Automated Code Review with Droid Exec

This tutorial shows you how to set up automated code review using Droid Exec in GitHub Actions. The workflow will analyze pull requests, identify issues, and post feedback as inline comments.

Unlike interactive CLI sessions, Droid Exec runs in headless mode, making it perfect for CI/CD automation. The agent analyzes code changes, identifies issues, and creates structured output that can be posted as PR comments.

How it works

The workflow:

Triggers on pull request events
Fetches the PR diff and existing comments
Uses Droid Exec to analyze the changes
Posts inline comments on specific lines with issues
Handles edge cases like duplicate comments and superseding outdated feedback

Droid Exec posting inline comments on PR with detected issues

Prerequisites

Before setting up the workflow, ensure you have:

A GitHub repository with Actions enabled
A Factory API key for authentication
Basic understanding of GitHub Actions

Configure authentication

Generate your Factory API key from the Factory Settings Page and add it as a repository secret:

Go to your repository’s Settings → Secrets and variables → Actions
Click “New repository secret”
Name: FACTORY_API_KEY
Value: Your Factory API key (starts with fk-)

Build the GitHub Actions workflow

Let’s build the workflow step by step to understand each component.

Set up the workflow trigger

Create .github/workflows/droid-code-review.yml and configure when it should run:

name: Droid PR Review

on:
  pull_request:
    types: [opened, synchronize, reopened, ready_for_review]

concurrency:
  group: droid-review-${{ github.event.pull_request.number }}
  cancel-in-progress: true

The concurrency group ensures only one review runs per PR at a time, canceling outdated runs when new commits are pushed.

Configure job permissions

Set up the job with necessary permissions and conditions:

jobs:
  review:
    runs-on: ubuntu-latest
    timeout-minutes: 30
    
    permissions:
      contents: read       # Read repository contents
      pull-requests: write # Post PR comments
      issues: write        # Update issue comments
    
    # Skip draft PRs to avoid noise during development
    if: github.event.pull_request.draft == false

    steps:

Checkout the repository

Add the checkout step to access the PR code:

      - name: Checkout PR head
        uses: actions/checkout@v4
        with:
          fetch-depth: 0  # Full history for accurate diffs
          ref: ${{ github.event.pull_request.head.sha }}

Install Droid CLI

Install the Factory Droid CLI in the runner:

      - name: Setup droid CLI
        run: |
          curl -fsSL https://app.factory.ai/cli | sh
          echo "$HOME/.local/bin" >> $GITHUB_PATH
          $HOME/.local/bin/droid --version

Prepare review context

Before running the review, we need to gather context about the PR. This step fetches existing comments and changed files:

      - name: Prepare context
        uses: actions/github-script@v7
        with:
          script: |
            const fs = require('fs');
            const path = '.git/pr_review';
            fs.mkdirSync(path, { recursive: true });

            // Fetch existing PR comments to avoid duplicates
            const prNumber = context.payload.pull_request.number;
            const comments = await github.paginate(github.rest.issues.listComments, {
              owner: context.repo.owner,
              repo: context.repo.repo,
              issue_number: prNumber,
              per_page: 100
            });
            fs.writeFileSync(`${path}/existing_comments.json`, JSON.stringify(comments, null, 2));

            // Fetch changed files with patches
            const files = await github.paginate(github.rest.pulls.listFiles, {
              owner: context.repo.owner,
              repo: context.repo.repo,
              pull_number: prNumber,
              per_page: 100
            });
            const reduced = files.map(f => ({ filename: f.filename, patch: f.patch }));
            fs.writeFileSync(`${path}/files.json`, JSON.stringify(reduced, null, 2));

            // Create diff via git locally
            const { execSync } = require('child_process');
            const base = context.payload.pull_request.base.sha;
            const head = context.payload.pull_request.head.sha;
            try {
              const diff = execSync(`git diff ${base}...${head}`, { 
                encoding: 'utf8', 
                stdio: ['ignore', 'pipe', 'pipe'] 
              });
              fs.writeFileSync(`${path}/diff.txt`, diff);
            } catch (e) {
              fs.writeFileSync(`${path}/diff.txt`, '');
            }

Configure the review agent

Now let’s create the review step. The prompt defines what issues to look for and how to format comments:

      - name: Generate inline review
        env:
          FACTORY_API_KEY: ${{ secrets.FACTORY_API_KEY }}
        timeout-minutes: 10
        run: |
          # Copy context files to current directory for droid to access
          cp .git/pr_review/existing_comments.json ./existing_comments.json
          cp .git/pr_review/files.json ./files.json
          cp .git/pr_review/diff.txt ./diff.txt

          echo "Files prepared:"
          ls -la *.json diff.txt 2>/dev/null || true

          cat > prompt.txt << 'EOF'
          You are an automated code review system. Review the PR diff and identify clear issues that need to be fixed.

          Input files (already in current directory):
          - diff.txt: the code changes to review
          - files.json: file patches with line numbers for positioning comments
          - existing_comments.json: skip issues already mentioned here

          Task: Create comments.json with this exact format:
          '[{"path": "path/to/file.js", "position": 42, "body": "Your comment here"}]'

          Focus on these types of issues:
          - Dead/unreachable code (if (false), while (false), code after return/throw/break)
          - Broken control flow (missing break in switch, fallthrough bugs)
          - Async/await mistakes (missing await, .then without return, unhandled promise rejections)
          - Array/object mutations in React components or reducers
          - UseEffect dependency array problems (missing deps, incorrect deps)
          - Incorrect operator usage (== vs ===, && vs ||, = in conditions)
          - Off-by-one errors in loops or array indexing
          - Integer overflow/underflow in calculations
          - Regex catastrophic backtracking vulnerabilities
          - Missing base cases in recursive functions
          - Incorrect type coercion that changes behavior
          - Environment variable access without defaults or validation

          Comment format:
          - Clearly describe the issue: "This code block is unreachable due to the if (false) condition"
          - Provide a concrete fix: "Remove this entire if block as it will never execute"
          - When possible, suggest the exact code change:
            ```suggestion
            // Remove the unreachable code
            ```
          - Be specific about why it's a problem: "This will cause a TypeError if input is null"
          - No emojis, just clear technical language

          Skip commenting on:
          - Code style, formatting, or naming conventions
          - Minor performance optimizations
          - Architectural decisions or design patterns
          - Features or functionality (unless broken)
          - Test coverage (unless tests are clearly broken)

          Position calculation:
          - Use the "position" field from files.json patches
          - This is the line number in the diff, not the file
          - Comments must align with exact changed lines only

          Output: 
          - Empty array [] if no issues found
          - Otherwise array of comment objects with path, position, body
          - Each comment should be actionable and clear about what needs to be fixed
          EOF

          # Run droid exec with stderr captured
          echo "Running droid exec..."
          droid exec --output-format debug --auto medium --model gpt-5-codex -f prompt.txt 2>&1

          # Check if comments.json was created
          if [ ! -f comments.json ]; then
            echo "❌ ERROR: droid exec did not create comments.json"
            echo "This indicates a failure in the review generation"
            exit 1
          fi

          echo "✅ comments.json created successfully"
          echo "=== Contents of comments.json ==="
          cat comments.json

Handle existing comments

To avoid confusion, supersede old “no issues” comments when new issues are found:

      - name: Supersede prior no-issues comments if new issues found
        if: always()
        uses: actions/github-script@v7
        with:
          script: |
            const fs = require('fs');
            if (!fs.existsSync('comments.json')) return;

            const comments = JSON.parse(fs.readFileSync('comments.json','utf8'));
            // Only supersede if we have actual issues to report
            if (!Array.isArray(comments) || comments.length === 0) return;

            const prNumber = context.payload.pull_request.number;
            const existingComments = await github.paginate(github.rest.issues.listComments, {
              owner: context.repo.owner,
              repo: context.repo.repo,
              issue_number: prNumber,
              per_page: 100
            });

            for (const c of existingComments) {
              const body = (c.body || '').trim();
              if (/^(✅\s*no issues|no issues found|lgtm)/i.test(body)) {
                await github.rest.issues.updateComment({
                  owner: context.repo.owner,
                  repo: context.repo.repo,
                  comment_id: c.id,
                  body: '[Superseded by new findings]'
                });
              }
            }

Submit the review

Post the generated comments as an inline review on the PR:

      - name: Submit inline review
        uses: actions/github-script@v7
        with:
          script: |
            const fs = require('fs');
            const prNumber = context.payload.pull_request.number;
            if (!fs.existsSync('comments.json')) {
              core.info('comments.json missing; skipping');
              return;
            }
            const comments = JSON.parse(fs.readFileSync('comments.json','utf8'));
            if (!Array.isArray(comments) || comments.length === 0) {
              // Post a single minimal top-level comment only if none exists yet
              const existing = await github.paginate(github.rest.issues.listComments, {
                owner: context.repo.owner,
                repo: context.repo.repo,
                issue_number: prNumber,
                per_page: 100
              });
              const hasNoIssues = existing.some(c => /no issues/i.test(c.body || ''));
              if (!hasNoIssues) {
                await github.rest.pulls.createReview({
                  owner: context.repo.owner,
                  repo: context.repo.repo,
                  pull_number: prNumber,
                  event: 'COMMENT',
                  body: '✅ No issues found in the current changes.'
                });
              }
              return;
            }
            const summary = 'Automated review with inline comments.';
            await github.rest.pulls.createReview({
              owner: context.repo.owner,
              repo: context.repo.repo,
              pull_number: prNumber,
              event: 'COMMENT',
              body: summary,
              comments: comments
            });

Save artifacts for debugging

Store review artifacts for troubleshooting:

      - name: Upload artifacts
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: pr-review-${{ github.event.pull_request.number }}
          path: |
            comments.json
            .git/pr_review/*.json
            .git/pr_review/diff.txt
            droid-exec-debug.log
          if-no-files-found: ignore
          retention-days: 14

Complete workflow file

Here’s the full workflow combining all the components:

name: Droid PR Review
on:
  pull_request:
    types: [opened, synchronize, reopened, ready_for_review]

concurrency:
  group: droid-review-${{ github.event.pull_request.number }}
  cancel-in-progress: true

jobs:
  review:
    runs-on: ubuntu-latest
    timeout-minutes: 30
    permissions:
      contents: read
      pull-requests: write
      issues: write
    if: github.event.pull_request.draft == false

    steps:
      - name: Checkout PR head
        uses: actions/checkout@v4
        with:
          fetch-depth: 0
          ref: ${{ github.event.pull_request.head.sha }}

      - name: Setup droid CLI
        run: |
          curl -fsSL https://app.factory.ai/cli | sh
          echo "$HOME/.local/bin" >> $GITHUB_PATH
          $HOME/.local/bin/droid --version

      - name: Prepare context
        uses: actions/github-script@v7
        with:
          script: |
            const fs = require('fs');
            const path = '.git/pr_review';
            fs.mkdirSync(path, { recursive: true });

            // Fetch existing PR comments
            const prNumber = context.payload.pull_request.number;
            const comments = await github.paginate(github.rest.issues.listComments, {
              owner: context.repo.owner,
              repo: context.repo.repo,
              issue_number: prNumber,
              per_page: 100
            });
            fs.writeFileSync(`${path}/existing_comments.json`, JSON.stringify(comments, null, 2));

            // Fetch changed files with patches
            const files = await github.paginate(github.rest.pulls.listFiles, {
              owner: context.repo.owner,
              repo: context.repo.repo,
              pull_number: prNumber,
              per_page: 100
            });
            const reduced = files.map(f => ({ filename: f.filename, patch: f.patch }));
            fs.writeFileSync(`${path}/files.json`, JSON.stringify(reduced, null, 2));

            // Create diff via git locally
            const { execSync } = require('child_process');
            const base = context.payload.pull_request.base.sha;
            const head = context.payload.pull_request.head.sha;
            try {
              const diff = execSync(`git diff ${base}...${head}`, { encoding: 'utf8', stdio: ['ignore', 'pipe', 'pipe'] });
              fs.writeFileSync(`${path}/diff.txt`, diff);
            } catch (e) {
              fs.writeFileSync(`${path}/diff.txt`, '');
            }

      - name: Generate inline review
        env:
          FACTORY_API_KEY: ${{ secrets.FACTORY_API_KEY }}
        timeout-minutes: 10
        run: |
          # Copy context files to current directory for droid to access
          cp .git/pr_review/existing_comments.json ./existing_comments.json
          cp .git/pr_review/files.json ./files.json
          cp .git/pr_review/diff.txt ./diff.txt

          echo "Files prepared:"
          ls -la *.json diff.txt 2>/dev/null || true

          cat > prompt.txt << 'EOF'
          You are an automated code review system. Review the PR diff and identify clear issues that need to be fixed.

          Input files (already in current directory):
          - diff.txt: the code changes to review
          - files.json: file patches with line numbers for positioning comments
          - existing_comments.json: skip issues already mentioned here

          Task: Create comments.json with this exact format:
          '[{"path": "path/to/file.js", "position": 42, "body": "Your comment here"}]'

          Focus on these types of issues:
          - Dead/unreachable code (if (false), while (false), code after return/throw/break)
          - Broken control flow (missing break in switch, fallthrough bugs)
          - Async/await mistakes (missing await, .then without return, unhandled promise rejections)
          - Array/object mutations in React components or reducers
          - UseEffect dependency array problems (missing deps, incorrect deps)
          - Incorrect operator usage (== vs ===, && vs ||, = in conditions)
          - Off-by-one errors in loops or array indexing
          - Integer overflow/underflow in calculations
          - Regex catastrophic backtracking vulnerabilities
          - Missing base cases in recursive functions
          - Incorrect type coercion that changes behavior
          - Environment variable access without defaults or validation

          Comment format:
          - Clearly describe the issue: "This code block is unreachable due to the if (false) condition"
          - Provide a concrete fix: "Remove this entire if block as it will never execute"
          - When possible, suggest the exact code change:
            ```suggestion
            // Remove the unreachable code
            ```
          - Be specific about why it's a problem: "This will cause a TypeError if input is null"
          - No emojis, just clear technical language

          Skip commenting on:
          - Code style, formatting, or naming conventions
          - Minor performance optimizations
          - Architectural decisions or design patterns
          - Features or functionality (unless broken)
          - Test coverage (unless tests are clearly broken)

          Position calculation:
          - Use the "position" field from files.json patches
          - This is the line number in the diff, not the file
          - Comments must align with exact changed lines only

          Output: 
          - Empty array [] if no issues found
          - Otherwise array of comment objects with path, position, body
          - Each comment should be actionable and clear about what needs to be fixed
          EOF

          # Run droid exec with stderr captured
          echo "Running droid exec..."
          droid exec --output-format debug --auto medium --model gpt-5-codex -f prompt.txt 2>&1

          # Check if comments.json was created
          if [ ! -f comments.json ]; then
            echo "❌ ERROR: droid exec did not create comments.json"
            echo "This indicates a failure in the review generation"
            exit 1
          fi

          echo "✅ comments.json created successfully"
          echo "=== Contents of comments.json ==="
          cat comments.json

      - name: Supersede prior no-issues comments if new issues found
        if: always()
        uses: actions/github-script@v7
        with:
          script: |
            const fs = require('fs');
            if (!fs.existsSync('comments.json')) return;

            const comments = JSON.parse(fs.readFileSync('comments.json','utf8'));
            // Only supersede if we have actual issues to report
            if (!Array.isArray(comments) || comments.length === 0) return;

            const prNumber = context.payload.pull_request.number;
            const existingComments = await github.paginate(github.rest.issues.listComments, {
              owner: context.repo.owner,
              repo: context.repo.repo,
              issue_number: prNumber,
              per_page: 100
            });

            for (const c of existingComments) {
              const body = (c.body || '').trim();
              if (/^(✅\s*no issues|no issues found|lgtm)/i.test(body)) {
                await github.rest.issues.updateComment({
                  owner: context.repo.owner,
                  repo: context.repo.repo,
                  comment_id: c.id,
                  body: '[Superseded by new findings]'
                });
              }
            }

      - name: Submit inline review (no gh CLI)
        uses: actions/github-script@v7
        with:
          script: |
            const fs = require('fs');
            const prNumber = context.payload.pull_request.number;
            if (!fs.existsSync('comments.json')) {
              core.info('comments.json missing; skipping');
              return;
            }
            const comments = JSON.parse(fs.readFileSync('comments.json','utf8'));
            if (!Array.isArray(comments) || comments.length === 0) {
              // Post a single minimal top-level comment only if none exists yet
              const existing = await github.paginate(github.rest.issues.listComments, {
                owner: context.repo.owner,
                repo: context.repo.repo,
                issue_number: prNumber,
                per_page: 100
              });
              const hasNoIssues = existing.some(c => /no issues/i.test(c.body || ''));
              if (!hasNoIssues) {
                await github.rest.pulls.createReview({
                  owner: context.repo.owner,
                  repo: context.repo.repo,
                  pull_number: prNumber,
                  event: 'COMMENT',
                  body: '✅ No issues found in the current changes.'
                });
              }
              return;
            }
            const summary = 'Automated review with inline comments.';
            await github.rest.pulls.createReview({
              owner: context.repo.owner,
              repo: context.repo.repo,
              pull_number: prNumber,
              event: 'COMMENT',
              body: summary,
              comments: comments
            });

      - name: Upload review-only artifacts
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: pr-review-only-${{ github.event.pull_request.number }}
          path: |
            comments.json
            .git/pr_review/*.json
            .git/pr_review/diff.txt
            droid-exec-debug.log
          if-no-files-found: ignore
          retention-days: 14

Test your reviewer

Create a test PR with some intentional issues to verify the workflow:

// Example code with issues
function processData(data) {
  if (false) {  // Dead code
    console.log("This never runs");
  }
  
  data.forEach(async item => {  // Missing await
    processItem(item);
  });
  
  if (data = null) {  // Changed assignment to comparison
    return;
  }
}

The reviewer should identify these issues and post inline comments.

Troubleshooting

Comments not appearing

Check the artifacts for comments.json to see if issues were found. Verify the position values match the diff line numbers.

Duplicate comments

The workflow checks existing_comments.json to avoid duplicates. Ensure this file is being created and passed to Droid.

Next steps

Customize the review criteria for your team’s standards
Add blocking reviews that prevent merging if critical issues are found
Integrate with other workflows like CI failure fixing
Create project-specific review profiles for different file types

Getting Started

User Guides

Configuration

Droid Exec (Headless)

Account

Automated Code Review with Droid Exec

How it works

Prerequisites

Configure authentication

Build the GitHub Actions workflow

Set up the workflow trigger

Configure job permissions

Checkout the repository

Install Droid CLI

Prepare review context

Configure the review agent

Handle existing comments

Submit the review

Save artifacts for debugging

Complete workflow file

Test your reviewer

Troubleshooting

Comments not appearing

Duplicate comments

Next steps

Getting Started

User Guides

Configuration

Droid Exec (Headless)

Account

​How it works

​Prerequisites

​Configure authentication

​Build the GitHub Actions workflow

​Set up the workflow trigger

​Configure job permissions

​Checkout the repository

​Install Droid CLI

​Prepare review context

​Configure the review agent

​Handle existing comments

​Submit the review

​Save artifacts for debugging

​Complete workflow file

​Test your reviewer

​Troubleshooting

​Comments not appearing

​Duplicate comments

​Next steps

How it works

Prerequisites

Configure authentication

Build the GitHub Actions workflow

Set up the workflow trigger

Configure job permissions

Checkout the repository

Install Droid CLI

Prepare review context

Configure the review agent

Handle existing comments

Submit the review

Save artifacts for debugging

Complete workflow file

Test your reviewer

Troubleshooting

Comments not appearing

Duplicate comments

Next steps