Assessment Bundle Governance Overview

Template (Existing Bundle)

Assessment Identifier(s)



Assessment Family



Assessment Score Method Descriptors



Hierarchy



Reasoning



Proposed Updates



Template (New Bundle)

Assessment Identifier(s)



Assessment Family



Assessment Score Method Descriptors



Hierarchy



StudentAssessmentEducationOrganizationAssociation



Reasoning



Existing Bundles

ACCESS (& ACCESS ALT)

Assessment Identifier(s)

  • ACCESS
  • Alternate-ACCESS

Assessment Family

  • None yet

Assessment Score Method Descriptors

  •  https://github.com/edanalytics/earthmover_edfi_bundles/blob/6b7520dcb10a2056280436766a14da14f281fb00/assessments/ACCESS/seeds/assessmentReportingMethodDescriptors.csv 

Hierarchy

  • Overall results are captured at the overall student assessment level and the four language domain results (such as Listening, Speaking, etc) are captured at the student objective assessment level.
graph TD;
ACCESS_Overall-->Listening;
ACCESS_Overall-->Speaking;
ACCESS_Overall-->Reading;
ACCESS_Overall-->Writing;
ACCESS_Overall-->Oral_Language;
ACCESS_Overall-->Literacy;
ACCESS_Overall-->Comprehension;

Reasoning

  • Because results exist at the composite level, this data model accurately reflects the structure of the assessment.

Proposed Updates

  • Update assessment_family to include either 'ACCESS' or 'Alternate-ACCESS' .
  • Populate school associations using the School Number field via the new StudentAssessmentEducationOrganizationAssociation entity.


ACT

Assessment Identifier(s)

  • ACT

Assessment Family

  • ACT

Assessment Score Method Descriptors

  •  https://github.com/edanalytics/earthmover_edfi_bundles/blob/e6a85a86bca52600dbc0a741acac1e4f86396086/assessments/ACT/seeds/assessmentReportingMethodDescriptors.csv 

Hierarchy

  • The composite results are captured at the overall student assessment level and all other subject-specific results are captured in the student objective assessment level, with parent objective assessments identified when that additional hierarchy exists. See the diagram below for more information:

Reasoning

  • The current combination of a single assessment identifier with a single composite subject and objective assessments with the subject-specific results correctly match the structure of the ACT assessment.
  • This structure also tentatively matches the structure of ACT's native Ed-Fi integration.

Proposed Updates

  • Populate school associations using the HS_Code field via the new StudentAssessmentEducationOrganizationAssociation entity.


CogAT

Assessment Identifier(s)

  • CogAT

Assessment Family

  • None yet

Assessment Score Method Descriptors

  •  https://github.com/edanalytics/earthmover_edfi_bundles/blob/main/assessments/CogAT/seeds/assessmentReportingMethodDescriptors.csv 

Hierarchy

  • The 3-way composite results are captured at the overall student assessment level and the other six scores (verbal, quantitative, nonverbal, VxQ, VxN, QxN) are captured at the student objective assessment level.
graph TD;
VxQxN-->Verbal;
VxQxN-->Nonverbal;
VxQxN-->Quantitative;
VxQxN-->VxQ;
VxQxN-->VxN;
VxQxN-->QxN;

Reasoning

  • Because results exist at the composite level, this data model accurately reflects the structure of the assessment.

Proposed Updates

  • Update assessment_family to include 'CogAT' .


DIBELS

Assessment Identifier(s)

  • dibels-8
  • dibels-next

Assessment Family

  • None yet

Assessment Score Method Descriptors

  •  https://github.com/edanalytics/earthmover_edfi_bundles/blob/main/assessments/Dibels_8_Benchmark/seeds/assessmentReportingMethodDescriptors.csv 

Hierarchy

This assessment includes composite scores and subscores from these categories, which are captured as objective assessments:
Title
Letter Naming Fluency
Phonemic Segmentation Fluency
Nonsense Word Fluency - Correct Letter Sounds
Nonsense Word Fluency - Words Recoded Correctly
Word Reading Fluency
Oral Reading Fluency - Accuracy
Oral Reading Fluency
Oral Reading Fluency - Error Rate
Reading Comprehension - Maze
Vocabulary
Spelling
RAN

Reasoning

  • Because this assessment is inherently single-subject and includes composite scores, this model matches governance standards. We accounted for changes in the assessment version with the different identifiers between Next and 8th edition.

Proposed Updates

  • Update assessment_family to include 'dibels'


IB

Assessment Identifier(s)

  • Assessment identifier concatenates IB with the SUBJECT field, i.e. IB - BIOLOGY.

Assessment Family

  • International Baccalaureate

Assessment Score Method Descriptors

  •  https://github.com/edanalytics/earthmover_edfi_bundles/blob/ed318effc144f9b208aeae49648d16ffaca881c3/assessments/IB/seeds/assessmentReportingMethodDescriptors.csv 

Hierarchy

  • We decided to make the grain of the assessment at the course level instead of the diploma level. The diploma results are included with each course level result.
  • In this case, there are no objective assessments.

Reasoning

  • Many students do not have diploma results, so if diploma results were captured at the overall student assessment level, they would have no scores to send.
  • We considered these three additional options in regards to the assessment identifier:
    Single IB assessment with diploma related results at the assessment level and courses as objective assessments.
    Drawbacks: for students who do not have diploma results (about half), the student assessment record would have no real scores.
    Assessment identifier includes the course title, with an additional IB assessment identifier for the diploma level results.
    Drawbacks: two different grains for the same assessment, difficult to associate diploma results with the relevant course results.
    Assessment identifiers which include course title are used for students with no diploma results; an additional IB assessment identifier is used for students with diploma results and courses are added as objective assessments.
    Drawbacks: two different grains for the same assessment, difficult to compare course level results for the two groups of students with the scores in different parts of the model.

Proposed Updates

  • None
  • We may be able to populate school associations using the SCHOOL_CODE field via the new StudentAssessmentEducationOrganizationAssociation entity. However, those codes appear to be inconsistent.


NWEA MAP Growth

Assessment Identifier(s)

  • NWEA-Map

Assessment Family

  • None yet

Assessment Score Method Descriptors

  •  https://github.com/edanalytics/earthmover_edfi_bundles/blob/main/assessments/NWEA_Map/seeds/assessmentReportingMethodDescriptors.csv 

Hierarchy

  • The overall results at each subject level are captured at the student assessment level and results by instructional area/goal are captured in the student objective assessment level.

Reasoning

  • Originally, our goal was to capture all of a student's results under a single assessment identifier (aka multiple student-level results (by subject) per administration linked to a single assessment). However, to match governance standards, we are proposing that we split the assessment by subject and link a single student-level result per administration to a single assessment. Any grouping can then be done using the assessment_family field.

Proposed Updates

  • Add subject directly to the assessment_identifier to match governance standards.
  • Also capitalize MAP, add 'Growth', and remove the vendor (will instead be captured exclusively by the namespace). New assessment identifiers would be:
  • MAP Growth Reading
  • MAP Growth Mathematics
  • MAP Growth Science
  • Update assessment_family to include 'MAP Growth' for grouping purposes.
  • Populate school associations using the School_StateID field (when filled in) via the new StudentAssessmentEducationOrganizationAssociation entity.


Renaissance Star

Assessment Identifier(s)

  • STAR-SM
  • STAR-SR
  • STAR-SEL

Assessment Family

  • None yet

Assessment Score Method Descriptors

  •  https://github.com/edanalytics/earthmover_edfi_bundles/blob/main/assessments/Renaissance_STAR/seeds/assessmentReportingMethodDescriptors.csv 

Hierarchy

  • The overall results at each subject level are captured at the overall student assessment level and multiple objective assessment types are captured at the student objective assessment level, with parent objective assessments identified when that additional hierarchy exists.
graph TD;
STAR_SR-->Domain_Groups_Reading;
Domain_Groups_Reading-->Dashboard_Standards_Reading;
Domain_Groups_Reading-->Skill_Areas_Reading;
STAR_SM-->Domain_Groups_Math;
Domain_Groups_Math-->Dashboard_Standards_Math;
Domain_Groups_Math-->Skill_Areas_Math;

Reasoning

  • The current combination of assessment identifiers split by subject and objective assessments correctly match the structure of the Star assessment.
  • The highest level at which scores exist is at the subject level, and no scores exist across the subjects.
  • Objective assessments are contained to a single subject.
  • These assessment identifiers already match governance standards.

Proposed Updates

  • Update assessment_family to include 'Star' for grouping purposes.
  • Populate school associations using the SchoolIdentifier field via the new StudentAssessmentEducationOrganizationAssociation entity.


SAT/PSAT

Assessment Identifier(s)

  • PSAT 8/9
  • PSAT 10
  • PSAT/NMSQT
  • SAT

Assessment Family

  • None yet

Assessment Score Method Descriptors

  •  https://github.com/edanalytics/earthmover_edfi_bundles/blob/main/assessments/PSAT_SAT/seeds/assessmentReportingMethodDescriptors.csv 

Hierarchy

  • The results at each section (math & reading/writing) are sent as separate student assessment records and the student objective assessments capture tests (reading, writing and language, and math) and subscores (Command of Evidence, etc).
  • The composite score is captured by each of the student assessment records.

Reasoning

  • Our initial understanding of the structure of the assessment lead us to believe we needed the overall student assessment results to capture the second level of the assessment (sections instead of the composite) in order to properly associate objective assessments. We have since determined this is not true.

Proposed Updates

  • Change the heirarchy to capture the composite scores at the student assessment level, and capture all other scores at the student objective assessment level. See this diagram for clarification on the proposed data model:
  • In the above scenario, the assessment identifiers would actually remain the same, but the section-level results would move to the student objective assessment level instead of being captured at the overall student assessment level.
  • Update assessment_family to include the relevant test type (PSAT 8/9 vs SAT, etc).
  • Populate school associations using the AI_CODE field via the new StudentAssessmentEducationOrganizationAssociation entity.


SC EOCEP

Assessment Identifier(s)

  • EOCEP-A
  • EOCEP-B
  • EOCEP-E2
  • EOCEP-H

Assessment Family

  • None yet

Assessment Score Method Descriptors

  •  https://github.com/edanalytics/earthmover_edfi_bundles/blob/feature/sc_eocep/assessments/EOCEP/seeds/assessmentReportingMethodDescriptors.csv 

Hierarchy

  • The overall results at each subject level are captured at the student assessment level and results by reporting type are captured in the student objective assessment level.

graph TD;
EOCEP_A-->Algebra;
EOCEP_A-->Functions;
EOCEP_H-->Expansion⋃
EOCEP_H-->Capitalism&Reform;

Reasoning

  • The current combination of assessment identifiers split by subject and objective assessments correctly match the structure of the SC EOCEP assessment.
  • The highest level at which scores exist is at the subject level, and no scores exist across the subjects.
  • Objective assessments/reporting method results are contained to a single subject.
  • These assessment identifiers already match governance standards.

Proposed Updates

  • Update assessment_family to include 'EOCEP'.
  • Populate school associations using the Tested School ID and Reported School ID fields via the new StudentAssessmentEducationOrganizationAssociation entity.


SC READY

Assessment Identifier(s)

  • SCREADY Statewide Summative Assessment

Assessment Family

  • None yet

Assessment Score Method Descriptors

  •  https://github.com/edanalytics/earthmover_edfi_bundles/blob/f96c428a09645245f1861f441a5dbf3a10d2e80a/assessments/DRC_SC_READY/seeds/assessmentReportingMethodDescriptors.csv 

Hierarchy

  • The overall results at each subject level are captured at the overall student assessment level and the subject reporting category/standards results are captured at the student objective assessment level.

Reasoning

  • Originally, our goal was to capture all of a student's results under a single assessment identifier (aka multiple student-level results (by subject) per administration linked to a single assessment). However, to match governance standards, we are proposing that we split the assessment by subject and link a single student-level result per administration to a single assessment. Any grouping can then be done using the assessment_family field.
  • The current hierarchy accurately reflects the hierarchy of the assessment.

Proposed Updates

  • Add subject directly to the assessment_identifier to match governance standards.
  • Also shorten the identifier:
  • SC READY ELA
  • SC READY Mathematics
  • SC READY Science
  • SC READY Social Studies (to capture results prior to 23-24)

graph TD;
SC_READY_ELA-->Language;
SC_READY_ELA-->Writing;
SC_READY_ELA-->Other_ELA_standatds;
SC_READY_MATH-->Geometry;
SC_READY_MATH-->Functions;
SC_READY_MATH-->Other_Math_standatds;
  • Update assessment_family to include 'SC READY' for grouping purposes.
  • Include additional score results as assessmentReportingMethodDescriptors.
  • We originally included only those necessary for current reporting needs, but we want to capture as many scores as possible.
  • Populate school associations using the SchoolID field via the new StudentAssessmentEducationOrganizationAssociation entity.


ACCESS/ACCESS Alt

Assessment Identifier(s)

  • ACCESS
  • Alternate-ACCESS

Assessment Family

  • None

Assessment Score Method Descriptors

  • Performance Level
  • Proficiency Level
  • Scale Score
  • Confidence Low Score
  • Confidence High Score
  • Cluster
  • Tier
  • Domain Termination
  • Status
  • Number of Correct Responses
  • Cue A Number Correct
  • Cue A Percent Correct
  • Cue B Number Correct
  • Cue B Percent Correct
  • Cue C Number Correct
  • Cue C Percent Correct

Hierarchy

  • The overall ACCESS/Alternative ACCESS results are captured at the overall student assessment level and the results for Listening, Speaking, Reading, Writing, Comprehension, Literacy, and Oral sections are captured at the student objective assessment level.
flowchart TD
A[ACCESS] --> B[Listening]
A --> C[Speaking]
A --> D[Reading]
A --> E[Writing]
A --> F[Comprehension]
A --> G[Literacy]
A --> H[Oral]

I[Alternate ACCESS] --> J[Listening]
I --> K[Speaking]
I --> L[Reading]
I --> M[Writing]
I --> N[Comprehension]
I --> O[Literacy]
I --> P[Oral]

Reasoning

  • The current combination of assessment identifiers at the overall ACCESS/Alternate ACCESS and objective assessments correctly match the structure of the assessment.
  • The highest level at which scores exist is at the overall level.
  • These assessment identifiers already match governance standards.

Proposed Updates

  • Add assessment family values of ACCESS & Alternate-ACCESS


New Bundles

South Carolina Alternate Assessments (SC-Alt)

Assessment Identifier(s)

  • SC-Alt ELA
  • SC-Alt Math
  • SC-Alt Science
  • SC-Alt Social Studies

Assessment Family

  • SC-Alt

Assessment Score Method Descriptors

  • Test Name
  • Scale Score
  • Scale Score Standard Error
  • Performance

Hierarchy

  • The overall results at each subject level will be captured at the overal student assessment level. No subscores exist.

StudentAssessmentEducationOrganizationAssociation

  • Populate school associations using the SchoolIdentifier field via the new StudentAssessmentEducationOrganizationAssociation entity.

Reasoning

  • These assessment identifiers would match governance standards. The split across these four subjects matches the split in  public analysis/aggregates  of the SC Alt data.


WIN

Assessment Identifier(s)

  • WIN_Soft_Skills (Title: WIN Work Ready Soft Skills Assessment)
  • WIN_Academic_Skills (Title: WIN Work Ready Academic Skills Assessment)
  • WIN_Soft_Skills_<2022

Assessment Family

  • WIN Work Ready

Assessment Score Method Descriptors

  • 9GR
  • CertLevel
  • Mode (will be captured as PlatformType instead of Score, technically)
  • Acc (will be captured as Accommodations instead of Score, technically)
  • Lev
  • RS
  • SS
  • Status
  • STime
  • FTime
  • Retake

Hierarchy

  • Soft Skills and Academic Skills will be treated as separate assessments, and the Math/Data/Reading Subtests will be captured as objective assessments under the Academic Skills assessment.

flowchart TD
A[Academic Skills] --> B[Math]
A --> C[Reading]
A --> D[Data]
G[Soft Skills]

StudentAssessmentEducationOrganizationAssociation

  • Populate school associations using the SchoolIdentifier field via the new StudentAssessmentEducationOrganizationAssociation entity.

Reasoning

  • The CertLevel score is a score that crosses the Math/Data/Reading subtests, so this accurately captures the highest level at which scores exist.
  • Soft skills & academic skills are considered the 'highest level' by the vendor:


TS Gold

Assessment Identifier(s)

  • GOLD_SE
  • GOLD_PHY
  • GOLD_LANG
  • GOLD_COG
  • GOLD_LIT
  • GOLD_MATH
  • GOLD_SCI
  • GOLD_ART
  • GOLD_ENG
  • GOLD_LANG_Spanish
  • GOLD_LIT_Spanish

Assessment Family

  • GOLD

Assessment Score Method Descriptors

  • Raw Score
  • Scale Score
  • WHE
  • NN
  • K-Readiness

Hierarchy

  • The overall results at each 'section' level are captured at the student assessment level and no additional scores are captured. Additional information about individual objectives does exist in the file but they are item-level scores and those are not currently being captured through this bundle because they would have to be captured as items instead of objective assessments, which we do not have downstream modeling built out for. We have also not heard of use-cases for these data, nor would they populate on Podium or Rally.

Reasoning

  • There are no scores at the overall GOLD level, and instead the highest level that scores are captured is at the 'Section' level (see screenshot below).


Mastery View Predictive Assessment

Assessment Identifier(s)

  • MVPA_{test_name}
  • This would lead to multiple assessment identifiers per 'subject'
  • e.g. MVPA_alg1, MVPA_algf, MVPA_eng1, MVPA_eng2, etc

Assessment Family

  • MVPA

Assessment Score Method Descriptors

  • PercCorr
  • AchLev
  • AchLevExt
  • SuggNMark
  • SuggLMark

Hierarchy

  • There do not appear to be composite scores across subjects, or any subscores lower than the individual test name (alg1, eng1, etc), there will only be overall assessments and no objective assessments.

Reasoning

  • Because the highest level at which scores are captured is the test name level (alg1, eng1, biol, etc.), then the proposed structure matched the governance standards.

Beacon

Assessment Identifier(s)

  • beacon_ela
  • beacon_math

Assessment Family

  • Beacon

Assessment Score Method Descriptors

  • Test Name
  • Scale Score
  • Scale Score Standard Error (SEM)
  • Theta
  • Theta SEM
  • Achievement Level

Hierarchy

  • The Math assessment can be decomposed in to 4 subcomponents:
  • Number and Quantity, Algebra, Measurement and Data, Geometry
  • The ELA assessment can be decomposed into 2 subcomponents, Reading and Writing, which each have further associated subcomponents:
  • Reading:
  • Key Ideas and Details, Craft Structure/Integration of Knowledge and Ideas, Vocabulary Acquisition and Use, Reading Text Types, Literary Text, Informational Text
  • Writing:
  • Text Types and Purposes, Conventions of Standard English, Research
  • NOTE: Listening is a possible subcomponent of the Beacon assessment, however SC does not require Listening to be administered. This subcomponent is not included in the bundle at present.
  • In addition to the more typical "Full Test", students may also take Beacon "Testlets", which are domain-specific.
  • Students may take a testlet for the Reading Section, individual Writing subcomponents, or individual Math subcomponents.
  • In these cases, Objective Assessment scores will be sent without a parent assessment.
  •  More information on Testlets 


StudentAssessmentEducationOrganizationAssociation

Students can be associated with schools by concatenating the District_Code and School_Code fields. This concatenation will result in a value conforming to the school_id used in edu.

Reasoning

  • Beacon is a formative assessment measuring student growth in ELA and Math from grades 3-8. Each assessment reports an overall scale score, as well as a performance level contextualizing performance relative to an established criterion.
  • Scores are provided at the composite levels of ELA and Math. Each respective section is comprised of multiple subcomponents, or objective assessment, scores. The proposed assessment structure accurately models the relationship between each assessment section and its subcomponents.