Ed-Fi Assessment Integrations using earthmover & lightbeam Documentation

This initial document provides an overview of Ed-Fi Assessment Integrations using earthmover and lightbeam and includes links to more detailed documentation on various topics.

A corresponding slide deck on these topics can be found  here .

Important Links





Ed-Fi Assessment Integration Process Overview:

When building out an integration for a new assessment (AKA an assessment that is not currently listed in the  bundles repository ), the general process is as follows:


1. Background Information

    .1Familiar yourself with the  earthmover  &  lightbeam  tools.
    .aThe READMEs of those repos contain all necessary documentation to install/use the tools.
    .bMultiple earthmover  example projects  exist to help familiarize yourself to its features.
    .2Review the documentation on the  Assessment domain  of the Ed-Fi data model.
    .aIt is impossible to map data to Ed-Fi without understanding the destination.
    .3Explore the  template bundle  to get a better sense of the structure/best-practices of an assessment bundle.
    .aThis bundle contains fake data and can be run in any python environment with earthmover installed.

2. Data Governance

    .1Read documentation on  🟢⁠⁠Assessment Data Governance in Ed-Fi⁠ .
    .2Gather relevant documentation about the assessment that you are working with. In order to properly map an assessment, you must deeply understand the structure and background information of that assessment. This documentation typically includes:
    .aSample files
    .bData dictionary
    .cTechnical reports from the vendor
    .3Fill out the  governance artifact template⁠  that will be reviewed by an assessment data governance committee.
    .aThe decisions here will impact how the specific assessment is modeled into the Ed-Fi standard, therefore must be done before any actual coding can occur.

3. Bundle Development

    .1Read documentation on  🟢⁠⁠Writing earthmover bundles to integrate assessment data into Ed-Fi⁠ .
    .2 Create a fork of the  earthmover_edfi_bundles  repository then duplicate the template bundle as a starting point for new assessment bundles.
    .3Update the template bundle code with necessary transformations, jinja logic, etc. for the given assessment.
    .aEnsure that data modeling matches the decisions from the governance artifact review.
    .4Test earthmover run to ensure no errors occur.
    .5Check that the resulting json files are valid.
    .aThis can be done in multiple ways, including  IDE extensions ,  jq , etc.
    .6(If possible) Manually run lightbeam  validate  against an ODS.
    .7Open a PR against the earthmover_edfi_bundles repository and tag Julianna Alvord for review (github user:@jalvord1).

4. Implementation of the Bundle

    .1Read documentation on  🟢⁠⁠Implementing earthmover bundles to load assessment data into Ed-Fi⁠ .
    .2Build out process to run earthmover & (depending on the circumstance) lightbeam.
    .aRun manually or built into a process that will run on a schedule?