Cloud Droid Environment (CDE) Initial Setup Guide

Droid performs best when it has access to the same tools your developers use. This guide outlines how to set up Cloud Droid Environments (CDE) for your Droids, significantly improving their coding abilities.

​

Requirements

For each repository where Droid will be installed, provide the following:

1. A devcontainer.json that installs dependencies and sets up a developer environment
2. Filepath filters for unit tests (e.g., *_test.py) and lintable files (e.g., *.py)
3. Terminal commands for:
   • Running a single unit test (e.g., pytest $file)
   • Generating a code coverage report (e.g., pytest --cov=. --cov-report=json $file)
   • Running linters and other validations on a single file (e.g., mypy $file)

Add these items to your repo’s .droid.yaml, under the “cloud_execution” settings.

​

Devcontainer Details

Droids use your Devcontainer to create a Cloud Environment for running tests, validators, linters, and code coverage. The container should mimic your local dev environment. Using your production Dockerfile as the dev environment may cause issues for Droid (as it would for a human developer).

​

Creating Your Devcontainer

​

Step 0: How Droid is Configured

• CDE installs an agent in an Amazon Linux VM that listens to commands from Droid.
• The VM loads your Factory platform secrets as environment variables.
• The agent starts your devcontainer.json using the Devcontainer CLI.
• The agent runs your unit tests, code coverage, and linters using devcontainer exec from the root of your repository.

​

Step 1: Install Packages & Libraries

In your devcontainer:

1. Install required binaries and libraries (e.g., apt-get install curl)
2. Install packages needed for tests, code coverage, linters, etc. (e.g., npm install, pip install -r requirements.txt)
3. Compile your code if needed (e.g., npx gulp build)

​

Step 2: Mock Dependencies

If your code has external dependencies (e.g., PostgreSQL, Redis), ensure they are properly mocked out for unit tests, or create local instances using Docker-Compose.

​

Testing Your Devcontainer

​

Step 0: Install the Devcontainer CLI

Refer to the wiki for installation instructions.

​

Step 1: Load Secrets

Load secrets as environment variables:

export SECRET_NAME=SECRET_VALUE

​

Step 2: Build Your Devcontainer

devcontainer up --workspace-folder /path/to/repo --config /path/to/devcontainer.json

​

Step 3: Test Commands

Check if your unit tests, code coverage, and linters can run:

devcontainer exec --workspace-folder <path_to_repo> <command>

Examples:

# JavaScript unit test
devcontainer exec --workspace-folder /path/to/repo npm run test -- foo.test.js

# Python linter
devcontainer exec --workspace-folder /path/to/repo mypy test.py

​

Important Notes

• The Devcontainer should be buildable locally, from outside your VPC.
• Mock out as many secrets and dependencies as possible.
• Aim to use public image registries. For private registries, try to bake commands directly into your Dockerfile.
• If any secrets or registries cannot be mocked, include details in the last section of this document.

​

Devcontainer Examples

Here’s an example devcontainer.json for a Node.js repo:

{
  "name": "Node.js",
  "image": "mcr.microsoft.com/devcontainers/javascript-node:20-bullseye"
}

For more examples, see Microsoft’s templates repo.

​

Uploading and Using Secrets

You can upload secrets to the Factory platform. They are fully encrypted and inaccessible from Factory’s end.

1. Navigate to “Integrations” and select your version control system.
2. Go to “Secrets” and select your repo.
3. Add secrets as needed.

Global secrets apply to every repo, while Repo-level secrets are specific. Repo-level secrets take precedence if specified at both levels.

​

Using Secrets in Devcontainers

Your devcontainer runs on a Factory cloud VM, which loads all secrets as environment variables. Common use cases:

1. Using secrets in Lifecycle Scripts:

{
  "image": "mcr.microsoft.com/devcontainers/javascript-node:20-bullseye",
  "initializeCommand": "docker login -u ${localEnv:USERNAME} -p ${localEnv:PASSWORD}"
}

2. Passing secrets as Dockerfile build args:

{
  "build": {
    "dockerfile": "/path/to/Dockerfile",
    "args": {
      "NPM_AUTH_TOKEN": "${localEnv:NPM_AUTH_TOKEN}"
    }
  }
}

3. Passing secrets as Docker-compose build args:

build:
  dockerfile: Dockerfile
  args:
    - GITHUB_ACCESS_TOKEN=${GITHUB_ACCESS_TOKEN}

​

Test Output Formats

• Unit Tests: Print results to stdout in JUnit XML format.
• Code Coverage: If supported, print results to stdout in Cobertura XML format.

​

Q&A

Q: What if my repo uses multiple testing frameworks? A: Provide necessary commands and filepath filters for each framework. Use a single devcontainer.json if possible, or separate ones if needed.

Q: How can I set up a mono-repo? A: Use path_filters in your .droid.yaml for each service, pointing to different Devcontainers and commands as needed.

​

Registries and Secrets

When setting up your Cloud Droid Environment, you may need to use private image registries or handle sensitive information. Here are some best practices:

​

Private Image Registries

1. If possible, use public image registries for your base images.
2. For private registries:
   • Use the initializeCommand in your devcontainer.json to log in to the registry:

     {
       "initializeCommand": "docker login -u ${localEnv:REGISTRY_USERNAME} -p ${localEnv:REGISTRY_PASSWORD} your-private-registry.com"
     }
     

   • Ensure the necessary credentials are added as secrets in the Factory platform.

​

Handling Secrets

1. Never hard-code secrets in your devcontainer.json or Dockerfile.
2. Use the Factory platform to store all sensitive information.
3. Reference secrets in your devcontainer.json using the ${localEnv:SECRET_NAME} syntax.
4. For sensitive build arguments, pass them to your Dockerfile:

   {
     "build": {
       "dockerfile": "Dockerfile",
       "args": {
         "ACCESS_TOKEN": "${localEnv:ACCESS_TOKEN}"
       }
     }
   }
   

​

VPN Configuration

If your development environment requires VPN access:

1. Consult with the Factory team to discuss VPN integration options.
2. You may need to provide VPN configuration details and credentials as secrets.
3. The Factory team will work with you to ensure secure VPN access for your Cloud Droid Environment.

​

Linters and Formatters

Integrating linters and formatters into your Cloud Droid Environment ensures code quality and consistency. Here’s how to set them up:

1. Install linters and formatters in your devcontainer.json:

   {
     "postCreateCommand": "npm install -g eslint prettier"
   }
   

2. Add linter and formatter configurations to your repository (e.g., .eslintrc, .prettierrc).

3. In your .droid.yaml, specify commands for running linters and formatters:

   cloud_execution:
     validation_config:
       - path_filters: ["*.js", "*.ts"]
         devcontainer: ".devcontainer/devcontainer.json"
         lint_command: "eslint $file"
         format_command: "prettier --write $file"
   

4. For compiled languages, include compilation steps in your lint command:

   lint_command: "javac $file && checkstyle $file"
   

5. Ensure your linter output is in a parseable format (e.g., JSON or XML) for better integration with Factory’s systems.

Remember to test your linter and formatter commands locally within the devcontainer before deploying to ensure they work as expected.

​

FAQ, Troubleshooting, and Support

For any questions or issues encountered during setup, refer to the FAQ and troubleshooting guide.