With our approach to bundle versioning we strive to balance the needs of users with a realistic maintenance model. We utilize two different delivery mechanisms for bundle updates.
- should be pushed to all users and represent the new single source of truth.
- occur when the code both before and after the change must be maintained going forward
In theory, we can write bundle files in a maximally-flexible style that attempts to accommodate all states of the world over all time into a single piece of code. In practice, we know that bundles code can change in fundamental ways that would be infeasible (or simply impossible) to capture via conditional logic. Thus, some mechanisms need to exist outside of the code, including some acceptance of breaking changes and git-based versioning.
We also plan (for now) to keep all of our assessment bundles in a single repo. Separating each bundle into its own repo brings certain benefits, and if we had a robust service for managing package versions, this solution could work well for both developers and users. However, at present, bundle definitions are more alike than they are different, and so it is common to update them all at once, which can be done much more easily in a single repo. Also, the cost of maintaining many repos is high.
We plan to employ a different strategy for each of the three types of bundle changes we consider most likely. It is important to note that while each individual type of change occurs relatively rarely (on the timescale of a year), taken together they will add up to a more frequent cadence of updates. Treating them differently helps diffuse the pressure on any one mechanism and provides the best balance between usability and maintainability.
are updates to Ed-Fi or to the format of an individual assessment. Such changes are inevitable, but we expect them to be relatively infrequent and minor. They constitute versioned releases because users cannot be required to update their version of Ed-Fi, or of an assessment. Users can pass this external information to Earthmover at runtime, and eventually there will be more automated tooling to infer this information. Bundles that have undergone external changes will continue to use a single metadata file.
are breaking updates that affect how all bundles run. Updates to the functionality or norms of Earthmover, or to the structure of bundle metadata, are examples of structural changes. Although we expect these to be rare, they are unpredictable and too major to be handled within bundle code. edfi_assessment_bundles so that existing users can keep running their bundles. However, we will treat these releases as snapshots of legacy code and so will not apply future bugfixes to them. We do not expect new users to utilize bundles tagged this way; the most recent version of bundle code is always preferred.
are breaking updates to how an individual bundle behaves, irrespective of external factors such as the Ed-Fi version. Due in part to uncertainty over the frequency and nature of these changes, we propose a further subdivision in how we treat them:
- Fixes to developer-error bugs will be pushed without regard for backwards compatibility
- Updates that don't fix a specific bug, but which we believe improve the representativeness beyond the scope of differences of opinion will be treated as bugfixes and similarly pushed out without versioning
- Updates that don't fix a specific bug, and for which we believe different users may prefer different versions, will result in the creation of a new bundle. We can use project composition to minimize the amount of duplicated code arising from such a case.
We believe the above approach is robust against incorrect assumptions about the nature and frequency of bundle changes. For example, if structural changes turn out to be rare while logical changes are frequent and disruptive, we could split out each bundle into its own repository and use tags at a finer grain ("state of the bundle" instead of "state of the world"). Similarly, if external changes are significant enough to make bundle code unmaintainable, we could factor out those chunks of logic into their own bundles and link them via project composition.