Lesson 7: Multi-Skill Choreography: Decomposing and Recombining Boundaries and Arbitrating Trigger Conflicts
Lesson Objective: When you have more than one skill in your repository, you willDeciding When to Split and When to Merge, and set the description to "mutually exclusive"; if there are two candidates, there is oneArbitration Schedule...rather than relying on the principle of “whoever gets it into the directory first wins.” In the last lesson, you already used a test suite to perform regression testing on a single skill:https://school.myshirtai.com/?p=4048; This lesson builds on that, addressing situations where multiple skills compete for triggering and where their boundaries overlap. This isn’t just another reflection on “the importance of collaboration,” but rather a set of decoupling guidelines, mutual exclusion rules, arbitration tables, and conflict examples that I’ll be able to submit tonight.
Introduction
Please complete Lesson 6, “Regression for Skills Using a Test Set,” first:https://school.myshirtai.com/?p=4048. You must have at least two that can load skills and run. run_skill_eval.py(or equivalent), examples of false triggers and missed triggers from Lessons 4–6. If you’re missing earlier foundational material, please review: Lesson 1 https://school.myshirtai.com/archives/3378; Lesson 2 https://school.myshirtai.com/archives/3462; Lesson 3 https://school.myshirtai.com/archives/3521; Lesson 4 https://school.myshirtai.com/archives/3661; Lesson 5 https://school.myshirtai.com/?p=3948The
List of Materials: Write permissions or the ability to submit PRs; the evaluation directory can include additional "conflicting pairs" examples; the team agrees on a priority arbitration table (which can be placed in skills/README.md). If there are no actual dual-trigger logs, start by using the sentence from Lesson 6, “failures”—“A was called, but B ran”—and don’t make up a harmonious story out of thin air.
If the assessment hook for Lesson 6 can only return “whether triggered + skill name,” that’s sufficient for this lesson; if the hook can only return a boolean, please first add the skill name parsing—otherwise, conflicting expectations cannot be expressed. It’s more honest to document the “hook capability gap” as a blocker before starting Lesson 7 than to simply write out sample code.
Time Box Recommendation: Tonight, spend 90 minutes working on just one pair of overlapping skills; don’t try to resolve the entire backlog in one night. Only by successfully completing one pair can the guidelines hold up; if the entire schedule runs idle, it will only leave behind a markdown that no one has worked on.

Concept: In choreography, address the boundaries first, then discuss who is smarter
Lesson 6 ensures that the triggering of a single skill is testable. Lesson 7 adds:Even when both are "eligible," they may still both want to play at the same time. Organize your response to answer the following four questions: Should this be split into two Skills? Are there any contradictory statements in the description? Which one takes precedence in case of a conflict? Should the user be asked to clarify in case of a tie?
- Modularity is a product decision:Permissions, the cost of failure, and corpus overlap take precedence over “looking good in the directory.”
- Mutually exclusive—add to description:When stating what to do explicitly, use short sentences such as “Do not handle X / If you encounter Y, please use Z.”
- Arbitration Forms Enter the Warehouse:Versioning of priority and tie-breaking strategies; table modifications must undergo conflict evaluation.
- Sample conflicts for the Lesson 6 assessment:Both candidate sentences must have the expected skill; “Either is fine” is not allowed as the gold standard.
The official Agent Skills documentation emphasizes that the `description` is responsible for triggering actions; when multiple skills are involved, the boundaries between triggers must also be clearly defined. That covers the concepts; next, we’ll move directly to decomposition, composition, and arbitration.
Designing a system isn’t about creating another “super skill” and cramming all abilities into it. A “super skill” blurs the lines between permissions, evaluations, and accountability. If an arbitration table can clarify the matter, don’t risk using a longer description.
The relationship with Lesson 6 can be summarized in one sentence: Single-skill scenarios ensure that “the skill should or should not be triggered”; multi-skill sequences ensure that “which one of us should be triggered.” Both sets of examples must be included in CI; neither can be omitted.
Things to Do Tonight:List all Skills in the repository name As in the first sentence of the description, identify the pairs you suspect of overlapping; if there are more than two pairs, handle the pair with the greatest overlap first.
Step-by-Step Guide: Decomposition and Composition, Mutual Exclusion Descriptions, Arbitration Tables, and Conflict Examples

- When will it be demolished?:Different permission boundaries (read-only FAQs vs. writable support tickets); different consequences of failure (checking documentation vs. sending an email); and almost no overlap in trigger scenarios. After breaking them down, each has its own independent
evals/The - When to Merge:In the same field, sharing a large amount of
reference/; After unpacking, there were accidental mutual triggers in more than about 20% of the samples; in the users’ minds, it’s simply “one assistant’s capability.” Before merging, combine the evaluation sets and then delete the directories. - Write a description of mutual exclusion:Add 1–2 negative boundary sentences to each Skill. For example, the FAQ Skill should state, “Do not create or modify support tickets; please use ‘ticket-ops’ for support tickets”; the ‘ticket-ops’ Skill should state, “Do not provide the original text of product policies; please use ‘shop-faq’ for policies.” Both positive capability statements and negative boundary statements must be detected during evaluation.
- Fill out the arbitration form:Using the HTML table above as a starting point, replace the "skill" names with those relevant to your business, then insert it into
skills/ARBITRATION.md(File names can be customized); When changing the "description" in a PR, you must specify whether it affects the priority. - Add conflict examples:At least 5 "dual candidate" utterances for each pair of overlapping skills; expected to be unique.
expect_skill. Run the script for Lesson 6; both lights must be green before merging.
| Conflict Types | Default Arbitration | Review Expectations |
|---|---|---|
| Safe/Write-to-Disk vs. Read-Only Query | Read-only mode takes precedence; writing to disk requires explicit designation or a second confirmation. | You cannot trigger the "Write to Disk" skill unless you are targeted. |
| Semantic Overlap in the "Two Reads" Skill | Sort by business priority as shown in the arbitration table; otherwise, prompt the user to select | Skills must be unique; duplicate true values are not allowed. |
| The user has tagged "skill" | Priority is given to those called by name, unless access is denied due to permission restrictions. | Roll Call Phrase: expect_skill=Caller |
| A tie with no players called out | Clarify questions; do not proceed with side effects without authorization | expect_trigger=false or expect_skill is empty and logging is enabled |
# Example of evals/conflict_pairs.jsonl
{"id":"C-01","utterance":"Where is the return deadline listed? Also, please open a follow-up ticket for me","expect_trigger":true,"expect_skill":"shop-faq","notes":"Answer the policy first; To create a ticket, specify `ticket-ops`"}
{"id":"C-02","utterance":"Use ticket-ops to create a follow-up ticket for order 123","expect_trigger":true,"expect_skill":"ticket-ops","notes":"User specified a specific team"}
{"id":"C-03","utterance":"Please process my return for me","expect_trigger":false,"expect_skill":"","notes":"Unresolved: Clarify whether the user is inquiring about policy or requesting a ticket"}
Add one more question to the PR template: "Have the arbitration table and conflict example IDs been updated?" A pull request cannot be merged if this field is left blank. Ensure that local commands match the CI commands.
How to write a valid mutual exclusion statement: It must be flagged by the tester. Simply writing “Please choose carefully” is not sufficient. You must describe observable behavior, such as “If the user requests to create or modify a ticket, this Skill will not process the request.” Then, use a conflicting statement to verify that “ticket-ops” is the expected result.
When making decisions about merging or splitting modules, I’ll call a 15-minute meeting with the parties involved: the security lead has the power to veto any merge; the business lead has the power to veto any unnecessary split. Summarize the meeting minutes in three lines and post them to PR to avoid any post-meeting claims like, “I thought you agreed to split it.”
Conflict examples must be drawn from real user sentences. Minor rewriting and anonymization are permitted, but do not create fictional sentences. Fictional sentences test your imagination, not triggers.
Things to Do Tonight:Select the pair of skills with the greatest overlap, write three conflict test cases, and run the program until at least one of them results in a "red light"; then use mutual exclusion or an arbitration table to fix it until it turns "green"—to confirm that the access control system actually handles conflict sets.
The arbitration team noted that the example table is just a starting point; replace the terms with your actual business names: swap out “shop-faq” and “ticket-ops” for the real skill names. If the rule in the table stating “explicit mention required for disk writing” conflicts with the product’s “one-line ordering” feature, have the product manager sign off on changing the expected behavior—don’t leave it up to the model to guess.
Errors and Setbacks (At Least Three)
- description: List only positive skills:Two skills are competing for the same sentence using synonyms. Solution: Enforce negative boundary sentences; resolve conflicts by evaluating synonym coverage.
- The arbitration agreement was made only verbally in a chat:New employees have no guidelines to follow when merging the third skill. Action: Check into the repository; CI checks for the existence of the file.
- Example of a conflict where "expect" is written as "A or B":The test always shows a false green. Solution: The expectation must be unique; for a tie, use `expect_trigger=false`.
- Merged the Skill but forgot to merge the evals:The old ID is invalid. Action: The merge PR must include an evaluation migration checklist.
- The "Write Disk" skill still triggers during a tie:Security priority did not trigger the hook. Resolution: Hard block at the hook layer; add a hard failure to the evaluation: "Do not write to disk unless explicitly permitted."
Things to Do Tonight:I'm intentionally including a generic phrase like "Help me process a return" to see if the current network/dry run triggers the disk-writing skill; if it does, add a hard block first, then revise the text.
There is another type of silent failure: when two read-only Skills both “match,” the Assistant generates a combined response, and the user assumes it comes from an authoritative policy, when in fact half of it is derived from an outdated reference. The default setting in the arbitration table for this scenario should be “single source,” rather than “draw from both.” In the evaluation, such sentences should be expected to have a single Skill name.
If your runtime is temporarily unable to read the arbitration table file, at the very least, include the priority in the mutex of each Skill description and hard-code a blacklist for the disk-writing class in the hook. File-based arbitration is the target state; hard blocking is the fallback state. Do not deploy the third Skill without a fallback state.
Exercises
- Draw a list of warehouse skills, mark at least one pair of overlaps, and decide whether to split or merge them (provide 5 lines of reasoning).
- Add mutual exclusion blocks to both sides of this pair and submit a description diff.
- Write to the arbitration table + at least 5 entries
conflict_pairsExample: Run toSKILL_EVAL_OKThe - Added 1 entry on “user roll call” and 1 entry on “tie results requiring clarification” from the actual log.
Completion Criteria: Viewable only by coworkers skills/README With the arbitration table, we can determine who wins in the event of a conflict without having to message you privately.
After completing the exercise, go back and check Lesson 6: CI—see if the conflict set has been added to the same job. If you still need to run a second script manually, it means the access control narrative is still split; merge the commands first, then declare Lesson 7 complete.
Preview of the Next Lesson
Lesson 8 Preview: Integrating Multi-Skill Arbitration with Production Traces and Feedback—How to Generate Conflict Examples with a Single Click from “Dual Candidate” Logs, and How to Add an Observation Period to Disk-Writing Skills. In this lesson, we’ll first ensure that offline splitting and merging, along with the arbitration table, are running smoothly; without conflict warnings, online orchestration will only result in skills interrupting each other.
Before Lesson 8, you can start by manually closing the smallest loop: every Friday, copy three suspected double-candidate entries from the log, write them up as conflict examples, and run regression tests on Monday. This manual approach may be clunky, but it works; once automation is in place, you can replace the manual copying—don’t just sit around waiting.
Please also remind our security colleagues that using multiple skills in an orchestration can alter the attack surface—sentences in the deception model that explicitly mention disk-writing skills should be included in the conflict and security examples, rather than appearing only in Red Team chats. The orchestration table and the security table can cross-reference the same ID.
Table of Contents for This Series
- Lesson 1: Write Your First Claude Agent Skill That Triggers Automatically in 30 Minutes — https://school.myshirtai.com/archives/3378
- Lesson 2: Adding a Reference to Skill / Validation Loop for “Write the Plan First, Then Execute” — https://school.myshirtai.com/archives/3462
- Lesson 3: Scripted Skills and Security Audits — https://school.myshirtai.com/archives/3521
- Lesson 4: Iterating Using Real-World Failure Cases description — https://school.myshirtai.com/archives/3661
- Lesson 5: Integrating Skill with a Real Repository's CI / Team-Shared Directory — https://school.myshirtai.com/?p=3948
- Lesson 6: Performing Regression on a Skill Using a Validation Set — https://school.myshirtai.com/?p=4048
- Lesson 7: Multi-Skill Choreography: Decomposing and Recombining Boundaries and Arbitrating Trigger Conflicts (This Article)


Comment List (0):
Load More Comments Loading...