CONTEXT PLACEMENT FOR CODING AGENTS: ON THE RELATIVE YIELD OF AGENTS.MD, SKILLS, COMMENTS, TESTS, AND TYPES

ABSTRACT

WE PROPOSE A CONTROLLED STUDY OF CONTEXT PLACEMENT IN AGENT-DRIVEN SOFTWARE DEVELOPMENT. THE QUESTION IS NOT WHETHER AGENTS BENEFIT FROM CONTEXT; THAT IS ALREADY OBVIOUS. THE QUESTION IS WHERE DIFFERENT KINDS OF CONTEXT SHOULD LIVE IN ORDER TO MAXIMIZE FIRST-PASS TASK COMPLETION, MINIMIZE REVISION CHURN, AND REDUCE REASONING ERRORS. MODERN CODEBASES EXPOSE AGENTS TO MULTIPLE CONTEXT SURFACES AT ONCE: GLOBAL REPOSITORY INSTRUCTIONS SUCH AS AGENTS.MD, INDEXED PROCEDURAL KNOWLEDGE SUCH AS SKILL FILES, LOCAL INTENT SIGNALS IN COMMENTS AND JSDOC, BEHAVIORAL SPECIFICATION THROUGH TESTS, AND STRUCTURAL CONSTRAINTS THROUGH TYPES. THESE SURFACES ARE OFTEN TREATED AS INTERCHANGEABLE. WE BELIEVE THEY ARE NOT. WE WILL MEASURE AGENT PERFORMANCE ACROSS PAIRED TASKS IN WHICH THE SAME REPOSITORY KNOWLEDGE IS MADE AVAILABLE THROUGH DIFFERENT CONTEXT LAYERS, IN ISOLATION AND IN COMBINATION. WE EXPECT PERFORMANCE TO DEPEND LESS ON RAW CONTEXT VOLUME THAN ON CONTEXT PLACEMENT. MORE SPECIFICALLY, WE EXPECT GLOBAL INSTRUCTION FILES TO PERFORM BEST ON REPOSITORY-WIDE POLICY AND WORKFLOW TASKS, SKILL FILES TO PERFORM BEST ON REUSABLE MULTI-STEP PROCEDURES, COMMENTS TO PERFORM BEST ON LOCAL INTENT RECOVERY, TESTS TO PERFORM BEST ON BEHAVIORAL AMBIGUITY, AND TYPES TO PERFORM BEST ON STRUCTURAL AMBIGUITY. OUR GOAL IS NOT TO DECLARE A SINGLE WINNER. IT IS TO PRODUCE A MAP OF WHICH CONTEXT LAYERS DELIVER THE HIGHEST MARGINAL VALUE FOR WHICH TASK CLASSES.

HYPOTHESIS

AGENT PERFORMANCE DEPENDS LESS ON THE TOTAL AMOUNT OF AVAILABLE CONTEXT THAN ON WHETHER THE RELEVANT CONTEXT IS LOCATED AT THE RIGHT LAYER OF THE SYSTEM. WE HYPOTHESIZE THE FOLLOWING: AGENTS.MD WILL PROVIDE THE HIGHEST YIELD FOR GLOBAL REPOSITORY NORMS, ARCHITECTURAL POLICY, AND WORKFLOW CONSTRAINTS. SKILL FILES WILL PROVIDE THE HIGHEST YIELD FOR SPECIALIZED, REUSABLE, MULTI-STEP TASKS THAT BENEFIT FROM PROCEDURAL RETRIEVAL. COMMENTS AND JSDOC WILL PROVIDE THE HIGHEST YIELD FOR LOCAL INTENT RECOVERY AT API BOUNDARIES AND NON-OBVIOUS CONSTRAINTS. TESTS WILL PROVIDE THE HIGHEST YIELD WHEN EXPECTED BEHAVIOR IS THE PRIMARY SOURCE OF TASK AMBIGUITY. TYPES WILL PROVIDE THE HIGHEST YIELD WHEN INTERFACE SHAPE, DATA FLOW, EFFECT SURFACE, OR ERROR STRUCTURE IS THE PRIMARY SOURCE OF TASK AMBIGUITY. SECONDARILY, WE HYPOTHESIZE THAT MISPLACED CONTEXT WILL UNDERPERFORM CORRECTLY PLACED CONTEXT EVEN WHEN THE UNDERLYING INFORMATION IS HELD CONSTANT.

1. BACKGROUND

AS CODING AGENTS MOVE FROM PROMPTED ASSISTANTS TO PERSISTENT COLLABORATORS, REPOSITORIES ARE BEING QUIETLY REWRITTEN AROUND THEM. TEAMS NOW ADD ROOT-LEVEL INSTRUCTION FILES, SKILL REGISTRIES, SYMBOL-LOCAL JSDOC, HIGHER TEST COVERAGE, AND STRONGER TYPE SURFACES, OFTEN WITHOUT A CLEAR MODEL OF WHICH OF THESE SURFACES ACTUALLY HELPS.

THE DEFAULT TENDENCY IS TO ACCUMULATE CONTEXT. IF AN AGENT FAILS, THE RESPONSE IS TO ADD ANOTHER FILE, ANOTHER NOTE, ANOTHER RULE, ANOTHER EXAMPLE. THIS MAY IMPROVE PERFORMANCE. IT MAY ALSO CREATE CONTEXTUAL OVERLAP, RETRIEVAL NOISE, AND HIGHER INFERENCE COST. THE QUESTION IS NOT WHETHER CONTEXT EXISTS. THE QUESTION IS WHETHER IT IS LOCATED WHERE THE AGENT CAN USE IT WITH THE LEAST AMBIGUITY.

WE BELIEVE AGENT-FACING CONTEXT FALLS INTO FIVE DISTINCT CLASSES: GLOBAL POLICY: HOW THIS REPO WORKS, WHAT MUST NEVER HAPPEN, WHICH PATTERNS ARE REQUIRED. PROCEDURAL KNOWLEDGE: HOW TO EXECUTE KNOWN MULTI-STEP WORKFLOWS. LOCAL INTENT: WHY THIS SYMBOL, CONSTRAINT, OR EDGE CASE EXISTS. BEHAVIORAL SPECIFICATION: WHAT THE SOFTWARE MUST DO. STRUCTURAL SPECIFICATION: WHAT SHAPES, EFFECTS, AND INTERFACES ARE LEGAL. THESE CLASSES SHOULD NOT BE ASSUMED TO BELONG TO THE SAME MEDIUM.

2. METHOD

WE PROPOSE A TASK-REPLAY STUDY ACROSS MULTIPLE CODEBASES IN WHICH THE SAME TASK IS RUN UNDER DIFFERENT CONTEXT-PLACEMENT CONDITIONS. EACH TASK WILL BE CLASSIFIED INTO ONE OF FIVE PRIMARY TASK CLASSES: WORKFLOW / REPO OPERATION TASKS EXAMPLES: BUILD FIXES, CI ADJUSTMENTS, MONOREPO CONVENTIONS, RELEASE PROCEDURES. PROCEDURAL TASKS EXAMPLES: ADDING A NEW ENDPOINT VIA A KNOWN INTERNAL PATTERN, RUNNING A MIGRATION PLAYBOOK, FOLLOWING A REPEATED FRAMEWORK-SPECIFIC FLOW. API / LOCAL INTENT TASKS EXAMPLES: MODIFYING EXPORTED FUNCTIONS, CHANGING PUBLIC CONTRACTS, TOUCHING CODE WITH NON-OBVIOUS RATIONALE. BEHAVIORAL TASKS EXAMPLES: BUG FIXES WHERE EXPECTED OUTPUT OR EDGE-CASE BEHAVIOR IS THE MAIN SOURCE OF AMBIGUITY. STRUCTURAL TASKS EXAMPLES: TYPE-LEVEL CHANGES, DATA-SHAPE UPDATES, ERROR-PATH CONSISTENCY, EFFECT / DEPENDENCY SURFACE CHANGES. THE SAME TASK WILL BE RUN WITH DIFFERENT CONTEXT CONDITIONS, INCLUDING: - BASELINE MINIMAL CONTEXT - AGENTS.MD ONLY - SKILLS ONLY - COMMENTS / JSDOC ONLY - TESTS ONLY - TYPES ONLY - SELECTED COMBINATIONS WHERE POSSIBLE, THE SAME INFORMATION WILL BE RE-ENCODED ACROSS CONDITIONS SO THAT THE STUDY TESTS PLACEMENT, NOT JUST PRESENCE.

• *PRIMARY OUTCOME - AGENT TURNS TO FIRST ACCEPTABLE COMPLETION FIRST ACCEPTABLE COMPLETION IS DEFINED AS: 1. PASSING TARGET TESTS, 2. NO NEW HIGH-SEVERITY TYPE OR LINT FAILURES, 3. NO OBVIOUS VIOLATION OF TASK INTENT IN LIGHTWEIGHT HUMAN REVIEW.
• *SECONDARY OUTCOMES - FIRST-EDIT ERROR COUNT - TOTAL EDIT CHURN - WRONG-BUT-PLAUSIBLE DETOURS - REVIEWER-FLAGGED "INTENT UNCLEAR" NOTES - TASK COMPLETION TIME - FREQUENCY OF AGENT RULE OR WORKFLOW VIOLATIONS - CONTEXT TOKEN CONSUMPTION AND RETRIEVAL OVERHEAD

3. PREDICTIONS

WE EXPECT A DISTRIBUTED, TASK-DEPENDENT RESULT RATHER THAN A SINGLE WINNING CONTEXT FORM.

• *AGENTS.MD WILL OUTPERFORM OTHER SURFACES ON REPOSITORY-WIDE POLICY AND WORKFLOW TASKS.
• *SKILL FILES WILL OUTPERFORM OTHER SURFACES ON REUSABLE MULTI-STEP TASKS WITH KNOWN PROCEDURES.
• *COMMENTS AND JSDOC WILL OUTPERFORM GLOBAL FILES ON TASKS THAT REQUIRE RECOVERING LOCAL INTENT OR HIDDEN RATIONALE.
• *TESTS WILL OUTPERFORM PROSE CONTEXT ON TASKS WHERE THE PRIMARY UNCERTAINTY IS EXPECTED BEHAVIOR.
• *TYPES WILL OUTPERFORM PROSE CONTEXT ON TASKS WHERE THE PRIMARY UNCERTAINTY IS STRUCTURE, INTERFACE SHAPE, OR CONSTRAINT COMPATIBILITY.
• *MISPLACED CONTEXT WILL SHOW LOWER MARGINAL VALUE THAN CORRECTLY PLACED CONTEXT EVEN WHEN INFORMATION CONTENT IS HELD ROUGHLY CONSTANT.
• *COMBINED CONDITIONS WILL OFTEN OUTPERFORM SINGLE-LAYER CONDITIONS, BUT WITH DIMINISHING RETURNS ONCE THE CRITICAL AMBIGUITY HAS BEEN RESOLVED.

4. THREATS TO VALIDITY

WE NAME THE THREATS WE CAN ANTICIPATE. THE LIST IS NOT EXHAUSTIVE.

• *INFORMATION EQUIVALENCE IS IMPERFECT: THE SAME KNOWLEDGE MAY CHANGE SHAPE WHEN MOVED FROM A TEST TO A COMMENT OR FROM A SKILL TO AGENTS.MD.
• *CONTEXT SURFACES MAY BE COMPLEMENTS, NOT SUBSTITUTES: A WINNER FRAME MAY MISDESCRIBE THE TRUE RELATIONSHIP.
• *TASK CLASSIFICATION CONTAINS JUDGMENT: SOME TASKS STRADDLE MULTIPLE CLASSES.
• *MODEL-SPECIFIC INTERACTIONS: SOME AGENTS MAY USE GLOBAL OR LOCAL CONTEXT MORE EFFECTIVELY THAN OTHERS.
• *TOOLING DIFFERENCES: THE QUALITY OF RETRIEVAL, FILE SELECTION, AND CONTEXT WINDOW MANAGEMENT MAY CHANGE THE RESULTS.
• *REPO CULTURE EFFECTS: SOME CODEBASES MAY ALREADY EMBED STRONGER SIGNALS IN ONE SURFACE THAN ANOTHER.
• *HUMAN REVIEW RUBRIC: ACCEPTABLE COMPLETION REQUIRES FROZEN REVIEW CRITERIA TO AVOID POST HOC DRIFT.

5. CONTRIBUTION

THE PRIMARY CONTRIBUTION OF THIS STUDY IS NOT THE CLAIM THAT AGENTS NEED CONTEXT. THEY CLEARLY DO. THE CONTRIBUTION IS A MORE PRECISE MODEL OF WHERE DIFFERENT KINDS OF CONTEXT SHOULD LIVE. WE AIM TO PRODUCE:

• *A TAXONOMY OF AGENT-FACING CONTEXT LAYERS
• *AN INITIAL ESTIMATE OF THE MARGINAL YIELD OF AGENTS.MD, SKILLS, COMMENTS, TESTS, AND TYPES
• *A DECISION FRAMEWORK FOR CONTEXT PLACEMENT BY TASK CLASS
• *AN EVALUATION HARNESS FOR REPLAYING TASKS UNDER DIFFERENT CONTEXT CONDITIONS
• *A MORE DISCIPLINED ALTERNATIVE TO CONTEXT ACCUMULATION BY INSTINCT

6. STATUS

DESIGNING TASK TAXONOMY, CONTEXT-REENCODING PROTOCOL, AND EVALUATION HARNESS. PILOT STUDY EXPECTED SUMMER 2026. INITIAL TASKS WILL FOCUS ON TYPESCRIPT CODEBASES WITH CLEARLY SEPARABLE POLICY, PROCEDURAL, LOCAL-INTENT, TEST, AND TYPE SURFACES. THE HARNESS AND TASK-CLASS RUBRIC WILL BE PUBLISHED BEFORE FULL DATA COLLECTION.