FAQ

Can I delete an account, a data or a job?

Accounts, data, and any Toolkit resources can be deactivated unless the jobs. Nothing can be deleted.

Is there a way to list your images in the private registry?

Not yet.

Does data pull work like rsync and only pulls the changes? or does it always pull everything?

It currently pulls everything.

If you still need to use rsync:

eai job new --image registry.console.elementai.com/shared.image/sshd --data your-data:/data:ro --restartable
eai job port-forward --last 2222

Then, in another terminal, use rsync:

rsync -avzh _toolchain@localhost:222 /your/folder

Just be sure that you run this in a safe environment i.e. NOT from shared workstations. Anyone that has access to that 2222 port can connect to it i.e. there’s no authentication on the SSH tunnel.

Can I set the profile for a session or a script so that I don’t have to specify --profile on every command ?

Yes to do so set the environment variable EAI_PROFILE

$ export EAI_PROFILE=dev
$ eai job new -- echo 'Hello World'
$ eai job info --last

The two commands will use the dev profile.

How often are the backups made?

The backup are done every Friday. Contact IT if you need a backup to be restored.

How does scheduling work?

There are two key concepts to Toolkit scheduling: resource usage and accounts.

At a high level, the scheduler strategy is a per account resource-based fair scheduling. It means that the scheduling priority goes in reverse order with resource usage. The fewer resources an account is using (low account occupancy), the higher the priority. An account not running a single job has top priority.

On top of that a few things are taken into consideration:

1. Each user, and not account, has the right to launch a single interractive job, which will be given the top priority. This by-passes regular scheduling.

2. Per account, the scheduling is made with two criteria: #. Highest bid goes first. #. FIFO (First In First Out).

3. A Preemptible job from an account cannot preempt other jobs within the same account. However, replica jobs and interactive jobs are allowed to preempt other jobs from their own account.

4. If a node is reserved for a specific account, jobs from that account are given priority on the node. Any job from a different account may be preempted to meet the resource requirements. In this scenario, account occupancy is ignored.

Warning

Since scheduling is account based, if many users are launching jobs within the same account, it still has the weight of a single account, which often makes user feel their jobs are not fairly scheduled. There is currently no Toolkit configuration to deal with that situation.

Why a Fair Share Scheduling strategy?

This strategy was chosen for two of its benefits.

1. If an account with no jobs comes in after other accounts are using the cluster at 100% with hundreds of jobs in the queue, that new account doesn’t have to wait for the new jobs to run.

2. It’s an incentive to request the right amount of resources. If an account’s jobs are taking more resources than they should, whenever the cluster is full, this account has fewer jobs running in parallel than it would if they were configured to use fewer resources.

How does the AI Toolkit Assistant work?

The AI Toolkit Assistant is a hosted AI-powered tool that provides intelligent assistance for AI Toolkit users. All models are running inside Toolkit environment. We use Mistral-2501 as the language model, multi-qa-mpnet-base-dot-v1 for embeddings, and ColBERT for re-ranking.

1. Agent knowledge: - The agents have knowledge about Toolkit docs, sharepoint related to YUL 201. - The agents currently have access to cluster in super pod YUL 201. Access to YUL 101 is not yet available. - If you find any missing details or would like to add additional knowledge to the agents, please reach out to the AI Toolkit team.

2. Agent Data Retention: - Stores anonymous versions of conversations to improve documentation in the future. - Maintains episodic conversations for contextual answering within a session, which are deleted after use.

3. Limitations: - The agents can hallucinate and provide you with wrong answer for some questions. Always check the documentation of commands before executing. - The agents can’t execute commands or scripts on your behalf. - Guardrails are implemented using simple system prompts, so easy hacking.

4. Pro tip: - Treat the assistant as a developer and provide all information needed to resolve the issue. - Keep chat session focused on one topic. - If the agents hallucinate, you can always ask them to “use tools, followed by the question”.

5. MCP Server: - The MCP server can be accessed via the Windsurf integration. - It has access to public documentation only and cannot access private documentation or clusters that are available through the AI Toolkit Assistant. - MCP uses the models provided by the IDE, not the custom models available through the AI Toolkit Assistant.

   • Do the following to add MCP server to windsurf:

     1. Make sure you have cli located in the path /usr/local/bin/eai

     2. Add the following to windsurf mcp_config.json (please refer windsurf documentation on adding mcp server):

        {
          "mcpServers": {
            "search_docs": {
              "command": "sh",
              "args": [
                "-c",
                "npx mcp-remote https://assistant.console.elementai.com/search_docs/mcp/ --header \"Authorization:Bearer $(/usr/local/bin/eai login token 2>/dev/null || echo 'MISSING_TOKEN')\""
              ]
            }
          }
        }