Skip to main content
Do Two Observers Agree? Running an IOA Check on School Behavior Data
BCBA Resources

Do Two Observers Agree? Running an IOA Check on School Behavior Data

Before mid-year decisions, check that a second observer would record the same thing. How to calculate each IOA method, what benchmark to borrow, and what to do when agreement is low.

BCBAPsychSpecialist5 references2 resources
Back to Blog
Bobby Hillman
Bobby Hillman, M.Ed., LBA, BCBA
Founder, Classroom Pulse
November 10, 2026
10 min read

Best For

Teams this article is built to help

BCBAPsychSpecialist

Category: BCBA Resources

Evidence

What backs this guide

5

Curated references are cited at the end of the article.

Materials

What you can leave with

3
  • Condensed key takeaways
  • 2 bonus downloads
  • Interactive self-check quiz
Share this article:

Most teams now have a full grading period of behavior data, and that data is about to be used: progress reports, winter IEP meetings, and the first real decision about whether a plan is working. Before the numbers carry that weight, it is worth asking one plain question. If a second trained adult had watched the same student at the same time, would they have recorded the same thing? An interobserver agreement (IOA) check answers that question, and a teacher and a paraprofessional can run one in a single class period.

What IOA Is and Why Schools Need It

Interobserver agreement is the degree to which two people, observing the same session independently, record the same behavior the same way. It is expressed as a percentage, and the way you calculate it depends on how the data were collected: counts, intervals, trials, or timed episodes.

In a school, IOA protects decisions. A frequency graph that drops after a new plan starts could mean the plan is working. It could also mean the para who collects most of the data has quietly started counting only the louder call-outs. Without a second observer, there is no way to tell those two stories apart. When a team is about to change services, fade a support, or defend an FBA, documented agreement is one of the strongest answers to the question every reviewer eventually asks. Our post on legally defensible behavior data explains why that record matters when a case is disputed.

How This Post Fits With Our Training Guide

If you are setting up a data system from scratch, start with training teachers to collect reliable behavior data. It covers decisions, definitions, training criteria, and when to schedule reliability checks. This post picks up where that one stops: how each IOA method is calculated, what a defensible benchmark looks like, how to run the check in a real classroom, and what to do with a disappointing number.

One distinction before the math. High agreement means two people applied the definition the same way. It does not prove that either of them was right, since two observers can be wrong about the same thing, or that the definition captures what the team actually cares about (Vollmer et al., 2008). Agreement is necessary for trustworthy data, not sufficient.

Count-Based Methods, With a Worked Example

Every example in this post except total duration uses a method built into the free IOA calculator, under the same names, so you can paste the numbers in and check the arithmetic yourself. The examples describe a hypothetical student and are not drawn from a real case.

Consider a fifth grader we'll call Nico, whose target behavior is calling out during a 30-minute math block. The teacher and the para both count call-outs and mark their tallies in six 5-minute intervals.

Interval 1 2 3 4 5 6 Total
Teacher 3 2 0 4 1 2 12
Para 3 2 0 3 0 2 10

Total count IOA divides the smaller total by the larger and multiplies by 100: 10 ÷ 12 × 100 = 83.3%. It is quick, and it is the least stringent index. Two observers can record entirely different call-outs and still produce matching totals.

Mean count-per-interval IOA does the same smaller-over-larger division inside each interval, then averages the results. Nico's intervals score 100, 100, 100, 75, 0, and 100, for a mean of 79.2%. Interval 5 scores zero because one observer counted a call-out and the other counted none.

Exact count-per-interval IOA counts only the intervals where both observers recorded exactly the same number. Four of six intervals match, so agreement is 66.7%. This is the most stringent index for count data.

Watch the Zero-Zero Interval

In interval 3, both observers recorded zero. Both count-per-interval methods treat that as full agreement. That is reasonable, but it means a session with long stretches of no behavior can look better than the observers' agreement on actual call-outs. Same session, three methods, three numbers (83%, 79%, 67%): report which method you used, every time.

Interval and Trial Methods

Interval recording (partial interval, whole interval, or momentary time sampling) produces a yes or no for each interval. If your team uses it, the interval recording template keeps both observers on the same grid.

Suppose the teacher and para use 30-second partial-interval recording for 10 minutes (20 intervals) to track Nico leaving his assigned area. The teacher scores the behavior in intervals 3, 9, and 15. The para scores it in intervals 3, 9, and 12. Every other interval is unscored by both.

  • Interval-by-interval IOA: agreements divided by all intervals. They agree on 18 of 20, so 90%.
  • Scored-interval (occurrence) IOA: only intervals where at least one observer scored the behavior (3, 9, 12, 15). They agree on 2 of those 4, so 50%.
  • Unscored-interval (nonoccurrence) IOA: only intervals where at least one observer scored no behavior. They agree on 16 of 18, so 88.9%.

Ninety percent sounds excellent. But when the behavior actually happened, these two observers agreed half the time. Interval-by-interval agreement often runs high, especially with partial-interval recording, and when little behavior occurs the easy agreements on empty intervals dominate (Reed & Azulay, 2011; Vollmer et al., 2008). That is why scored-interval IOA is the better index when the behavior is rare, and unscored-interval IOA is the better index when the behavior fills most intervals (Reed & Azulay, 2011).

Trial-by-trial IOA uses the same arithmetic for discrete opportunities. If both adults score whether a student followed a direction within the response window on 10 delivered directions, agreement is the number of trials scored the same way divided by 10.

Timed Methods: Duration and Latency

For behavior where length matters more than count, both observers time each episode independently. (If you are not sure whether a behavior calls for duration, frequency, or latency, see our guide to choosing between those measurements.) Say Nico has three out-of-seat episodes. The teacher times them at 120, 45, and 300 seconds; the para times them at 110, 60, and 290.

  • Total duration IOA: the smaller summed time over the larger. 460 ÷ 465 × 100 = 98.9%.
  • Mean duration-per-occurrence IOA: smaller over larger for each episode (91.7%, 75.0%, 96.7%), then averaged: 87.8%.

Total duration hides disagreements because a long episode one observer timed short can offset one they timed long; Reed and Azulay (2011) describe mean duration-per-occurrence as a much more conservative estimate. Mean latency-per-response IOA works the same way, pairing each observer's latency (for example, seconds from a direction to starting the task) and averaging the ratios. Total duration is easy to do by hand.

How Much Agreement Is Enough, and How Often to Check

There is no single required IOA standard for school teams. The guideline most behavior analysts learn comes from Cooper, Heron, and Heward (2020): at least 90 percent agreement for an established measure and at least 80 percent for a new one, with IOA collected for at least 20 percent of sessions, preferably 25 to 33 percent.

The clearest published bar for research comes from the What Works Clearinghouse, which reviews education research for the U.S. Department of Education, sets these requirements for direct-observation outcomes in single-case studies in its current Procedures and Standards Handbook, Version 5.0 (What Works Clearinghouse, 2022):

  • The behavior is measured by more than one assessor.
  • Agreement is collected in each phase, such as baseline and intervention.
  • Agreement data cover at least 20 percent of the data points.
  • Minimum values are .80 for percentage agreement, or .60 for Cohen's kappa.

The two line up: the research standard was written for reviews, not classrooms, and the Cooper guideline gives a practitioner the same floor with a stricter target once a measure is established. Both are defensible to borrow when data will support an FBA, a plan change, or a disputed decision. In practice: check in baseline and again after the plan starts, aim for at least one session in five and ideally one in three or four, and spread the checks across routines, days, and observers rather than clustering them in one good week.

Two cautions keep the number honest. First, there is no magic score: methods differ in how lenient they are, so a high percentage on a lenient index should not reassure you, while a low percentage on any index should concern you (Vollmer et al., 2008). The interval records for Nico above clear .80 on interval-by-interval agreement and fall far short on scored-interval agreement. Second, checks do not have to be elaborate. Vollmer and colleagues suggest that a practitioner could run a 10-minute monitoring session once a week per client, which a school-based BCBA can scale down to the few students whose data will drive the biggest decisions this term.

Running a Check With a Teacher and a Para

The most practical observer pair is usually already in the room. Here is a sequence that works without adding staff.

A Six-Step Classroom IOA Check

1. Pick a window when the behavior is likely

A 10- to 15-minute block where the behavior usually shows up, and where the second observer is not leading instruction. Scheduling observations when behavior is most likely is especially useful for low-rate behavior (Vollmer et al., 2008).

2. Put the same definition in both hands

Printed, with examples, non-examples, and a rule for unclear moments. Our para data quality tips show what a usable definition looks like.

3. Match the method and the clock

Same recording method, same interval length, same start and stop. Start together on a shared signal or timer so interval 4 means the same 30 seconds for both people.

4. Record independently

No conferring, no glancing at each other's sheets, no "did you get that one?" until both records are finished. Sit apart if the room allows.

5. Compare the same day

Calculate agreement, then walk through each disagreement while the session is fresh. The disagreements are where the training happens.

6. Keep the record

Date, observers, behavior, definition version, method, and the result. A percentage nobody wrote down cannot support a decision later.

Two school-specific notes. When the teacher is teaching, make the para the primary observer and bring in the BCBA, school psychologist, or case manager as the second observer for that window. And prefer familiar adults: a stranger with a clipboard can change how a student behaves, which makes the check less representative of an ordinary day.

For step 6, teams using Classroom Pulse on the BCBA Solo plan and above can enter both observers' interval records for the same session; the agreement is calculated and saved with the definition used, and the second observer does not need an account.

When Agreement Comes Back Low

A low number is information, not a verdict on anyone's competence. Start by looking at where the disagreements cluster, then work through the likely causes.

Likely Causes

  • The definition. Disagreements bunch up on borderline examples, such as whether a whispered answer counts as a call-out. Poorly phrased definitions are a common reason for poor reliability (Vollmer et al., 2008).
  • The clock. One observer started late, or intervals drifted out of alignment.
  • Episode boundaries. In duration data, the observers disagree about when an episode ends.
  • Observer drift. Over weeks, observers gradually change how they apply a definition, sometimes together (Kazdin, 1977).
  • Reactivity. Observers can record more carefully when they know they are being checked, so routine data may be weaker than check-day data (Kazdin, 1977; Vollmer et al., 2008).

What to Do

  • Add the disputed examples to the definition as examples or non-examples, and write a rule for unclear moments.
  • Use one shared timer or a vibrating interval prompt.
  • Write an end rule, for example "the episode ends after 10 seconds back in the assigned area."
  • Hold a short booster session where both observers score the same practice examples, then rotate observer pairs.
  • Make checks routine and spread across the term, so a checked session looks like any other session.

Then decide what the low number means for the data you already have. Do not quietly delete it. Note the low-agreement period on the graph or in the record, avoid high-stakes decisions that rest only on that stretch, and re-check after the fix. Data that two observers cannot agree on can make a plan look like it is working, or failing, when it is not. Vollmer and colleagues recommend reviewing reliability data before major clinical decisions.

Frequently Asked Questions

Can a teacher and a para who work together every day be independent observers?

Yes, if they record separately and do not confer until both records are done. Independence is about the recording, not the relationship. Because close colleagues can drift toward the same habits, pair each of them with someone from outside the classroom now and then.

How should IOA appear in an FBA or progress report?

Name the method, the number of sessions checked out of the total, the phases covered, and the mean and range of agreement. A sentence like "scored-interval agreement was checked in 4 of 18 sessions across baseline and intervention" tells the reader far more than a single percentage.

What if the behavior never happens during the check?

Scored-interval agreement cannot be calculated when neither observer scored an occurrence, and interval-by-interval agreement will look perfect without telling you much. Reschedule for a window when the behavior is more likely.

Can ABC data be checked for agreement?

Free-text narratives are hard to compare. If your ABC log uses preset antecedent and consequence categories, both observers can code the same incidents and you can compare category by category, the same way you would score trials.

Run One Check Before Winter Break

You do not need an IOA program to start. Pick the one student whose data will drive the biggest decision this winter, choose a 15-minute window, and have two adults record it independently. If agreement is strong, you can present the data with more confidence. If it is weak, you found out before the meeting instead of during it.

More resources for school-based behavior analysts are collected on our page for school-based BCBAs. When your two records are ready, run them through the calculator.

Open the Free IOA Calculator

References

Cooper, J. O., Heron, T. E., & Heward, W. L. (2020). Applied behavior analysis (3rd ed.). Pearson.

Kazdin, A. E. (1977). Artifact, bias, and complexity of assessment: The ABCs of reliability. Journal of Applied Behavior Analysis, 10(1), 141-150. https://doi.org/10.1901/jaba.1977.10-141

Reed, D. D., & Azulay, R. L. (2011). A Microsoft Excel 2010 based tool for calculating interobserver agreement. Behavior Analysis in Practice, 4(2), 45-52. https://doi.org/10.1007/BF03391783

Vollmer, T. R., Sloman, K. N., & St. Peter Pipkin, C. (2008). Practical implications of data reliability and treatment integrity monitoring. Behavior Analysis in Practice, 1(2), 4-11. https://doi.org/10.1007/BF03391722

What Works Clearinghouse. (2022). What Works Clearinghouse procedures and standards handbook, version 5.0. U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance. https://ies.ed.gov/ncee/wwc/Handbooks

Put This Into Practice

Turn the article into action with ready-to-use materials. Downloads stay open; emailed resources and quiz results are opt-in.

Key Takeaways

  • Interobserver agreement asks whether a second trained adult, recording independently at the same time, would have recorded the same thing.
  • The same session can produce different numbers by method, so always report which IOA method you used.
  • For rare behavior, scored-interval agreement is more honest than interval-by-interval agreement, which empty intervals can inflate.
  • The What Works Clearinghouse single-case standard (Version 5.0) asks for agreement in each phase, on at least 20 percent of data points, at .80 or higher.
  • When agreement is low, check the definition, the clock, episode boundaries, observer drift, and reactivity before trusting the data for a decision.

Bonus Materials

Clean downloads to pair with this article

Download resources directly or send a copy to your inbox. Marketing updates are optional.

2 ready-to-use downloads
PDF download

IOA Calculation Worksheets

Printable worksheets for working through agreement calculations by hand.

Download now
PDF download

Interval Recording Sheet

A shared interval grid so two observers record on the same timeline.

Download now

IOA Check Readiness Quiz

Four questions on calculating and interpreting interobserver agreement for school behavior data.

4 questions~2 min

For school-based BCBAs

Keep a Record of Every Agreement Check

On the BCBA Solo plan and above, you enter two observers' interval records from the same session and the agreement is calculated and saved with the definition used.

See Classroom Pulse for School-Based BCBAs

About the Author

Bobby Hillman
Bobby Hillman, M.Ed., LBA, BCBA
Founder, Classroom Pulse

Bobby Hillman is a Board Certified Behavior Analyst and Licensed Behavior Analyst with an M.Ed. in Special Education and more than 25 years in public special education and autism services. He works as a Behavior Instructional Specialist and founded Classroom Pulse to reduce the documentation load on classroom teams while keeping the detail FBA and BIP work requires.

About Classroom Pulse

Get More Insights Like This

One email on Fridays with the week's new articles. Unsubscribe anytime.

Stay updated with behavior tracking tips. Unsubscribe anytime. Privacy Policy.