IDEALIT Passport
Menu

Passport search

Find a mission or fix

Type at least two characters.

    Browse help by symptomBrowse all references
    How to start
    Curriculum previewThis is not your assigned passport. No identity, answers, or completion progress are stored here.How to start your passport
    Handbook / labs

    labs

    Lab: Manual Versus Agent-Assisted Work

    Goal: complete a second bounded FirstSteps change with an agent, verify it personally, and compare the work with the manual baseline.

    View exact sourceReturn to my dashboard

    Goal: complete a second bounded FirstSteps change with an agent, verify it personally, and compare the work with the manual baseline.

    Prerequisites:

    You do not need a paid AI service. Zed with personal OpenRouter is an optional alternative, not a passport requirement.

    Execution context: VS Code or the approved agent interface on your local computer, with the fictional IDEALLab/FirstSteps repository open. Do not run this exercise in a real research repository or on Euler.

    1. Establish A Clean Baseline#

    Run personally in the FirstSteps repository:

    git status --short --branch
git log --oneline -3
python first_steps/execute_me.py

    Expected result: the onboarding branch is not main, the manual change is committed, the working tree is clean, and the program runs.

    If the tree is not clean, inspect git status and git diff. Commit the understood manual task or remove only understood unwanted edits before starting the agent comparison.

    2. Start With Read-Only Inspection#

    Create a new agent session. Do not enable automatic approval for every command. Enter:

    Goal: plan one regression test for the current FirstSteps program.
Context: inspect README.md, pyproject.toml, first_steps/execute_me.py, and Git
status first.
Constraints: read only; do not edit, install, commit, push, or use the network.
Verification: identify the standard-library test command that would prove the
current printed output remains unchanged.
Output: list inspected files, proposed test behavior, expected file change, and
any uncertainty.

    Compare the plan with the actual files. Reject a plan that invents a dependency, changes application behavior, or touches unrelated configuration.

    3. Authorize One Bounded Edit#

    If the plan is appropriate, continue in the same session:

    Goal: add a standard-library unittest that proves the existing program prints
the three voltage values as a comma-separated line.
Context: use the files already inspected.
Constraints: create only tests/test_execute_me.py; do not change application
code, dependencies, configuration, Git history, or remote state; ask before any
terminal command.
Verification: python -m unittest discover -s tests; python first_steps/execute_me.py
Output: make the smallest test, then summarize the exact diff and limitations.

    Read every proposed terminal command before approving it. A familiar command is not automatically correct in the wrong repository or environment.

    4. Review Without Trusting The Summary#

    After the agent stops, personally run:

    git status --short
git diff --check
git diff

    git status should show ?? tests/ because the test is new and untracked. git diff does not display untracked file contents. Open tests/test_execute_me.py directly in the editor and review every line now; after staging in step 7, git diff --cached must show the complete new file.

    Confirm:

    • only tests/test_execute_me.py was added;
    • the test checks observable behavior rather than duplicating implementation;
    • no secret, data, cache, generated output, or editor configuration appears;
    • you can explain each import, assertion, and output capture.

    Reject or revise the change if any condition fails. Record at least one suggestion you accepted or rejected and why.

    5. Verify Personally#

    Run the declared checks yourself, even if the agent already ran them:

    python -m unittest discover -s tests
python first_steps/execute_me.py

    If the FirstSteps repository provides a reproducible development environment containing Ruff, mypy, and pre-commit, also run its documented checks. Do not install guessed global packages merely to expand this lab's evidence.

    Record the exact checks that ran and any failure. Do not report a command as passing because it appears only in the agent summary.

    6. Compare The Workflows#

    Add this evidence to the assessment or PR description without including chat transcripts, credentials, or unnecessary account information:

    Manual task approximate time:
Agent-assisted task approximate time:
Agent/interface used:
Commands proposed or run by the agent:
Suggestion accepted and why:
Suggestion rejected or revised and why:
Checks I personally ran:
Paid usage before/after: Not applicable, or sanitized amount:
One limitation, error, or suspicious behavior:
Which workflow required more review and why:

    The exercise does not assume that the agent is faster. Review and correction time is part of the comparison.

    7. Commit The Reviewed Test#

    Stage only the new test after verification:

    git add -- tests/test_execute_me.py
git diff --cached
git commit -m "test(firststeps): cover voltage output"
git push

    Update the existing draft FirstSteps PR. Do not merge it.

    Expected Result#

    • The manual change and agent-assisted test are separate understood commits.
    • The test and existing checks pass when run personally.
    • The PR contains only the intended training files.
    • The comparison identifies evidence, review work, and at least one limitation.

    Common Failures And Safe Recovery#

    • The agent edits application code: stop, inspect the diff, and restore only the unwanted application path. Ask for a test-only patch.
    • The agent adds pytest or another dependency: reject it; this exercise uses Python's standard library and does not require installation.
    • The repository asks for Ruff, mypy, or pre-commit but does not declare a reproducible development setup: report the training-repository defect and continue only with the standard-library verification above. Do not install guessed global packages merely to finish.
    • Two agent sessions edit the same folder: stop both, inspect Git state, keep one bounded task, and recover through reviewed diffs.
    • The agent committed or pushed unexpectedly: stop, preserve the branch, inspect git log and the remote PR, and ask the maintainer before rewriting history.
    • Protected information was entered: stop sharing and follow the incident procedure.

    Understand Before Accepting AI Output#

    • I completed and understood the manual workflow first.
    • I can distinguish the model, provider/gateway, harness, and interface used.
    • I reviewed every changed line and proposed command.
    • I personally ran and recorded the verification.
    • I can explain one accepted and one rejected/revised suggestion.
    • I know that speed is not evidence of correctness.

    Evidence#

    Submit the updated draft PR, both commit messages, the personally run checks, and the comparison fields above. A human reviewer may ask you to explain any line or command without the agent answering for you.

    Ask For Help When#

    The training baseline differs from the documented task, the agent cannot stay within one file, verification dependencies are undeclared, Git history changed unexpectedly, or recovery could discard manual work.