Metadata XML Transactions

Each transaction listed below has an XSD (stylesheet) associated with it defining the required and optional fields.

mifNew:A completely new component (dataset). This can be a new component within an existing collection, or a new component in a completely new collection.

mifNewInstance:  A new instance of an existing component (dataset). This xml stylesheet ONLY contains items that are discontinued, redefined, or brand new.

Required fields:  component, collection short name, collection long name, instance

1. If an item is not listed in the newInstance xml file then it will automatically continue to the new instance if its timeframe continues. 

·        Only if there is no break in series (the most current instance) the item will continue.  That means it is looking for a combination of both continues flag and most current instance.If there is a break in series then it will put a discontinue flag on the item and NOT continue the item to the new instance.

2. If you redefine an item, then it will discontinue the existing item with the same name (if it is not already discontinued) and then create a new item starting with the new instance.  This works the same same whether adding an instance after or before.

-         When going forward this means it puts a discontinue flag on the item’s timeframe and that timeframe does not get the new instance in the periodmap table.

-         When going backward (adding before first instance) there is NO discontinue flag, the item gets a different timeframe that does not include the new (earlier) instance in the periodmap table.

3. If you discontinue an item it will appropriately update the start/stop for the item. Discontinuing an item at the end of a series effectively changes the “stop” date.

Discontinuing an item at beginning of a series effectively changes the “start” date compared to all other items that continued back.

-         When going forward this means it puts a discontinue flag on the item’s timeframe and that timeframe does not get the new instance in the periodmap table.

-         When going backward (adding before first instance) there is NO discontinue flag, the item gets a different timeframe that does not include the new (earlier) instance in the periodmap table.

mifModify:  This makes any and all changes to existing information (dataset and item levels) for the instance specified ONLY.  (Note: some dataset level information is not instance-specific.)

1. update existing dataset level information

Required fields:  component, collection short name, collection long name, instance

Optional fields that can be changed (changes apply to all instances):

• Collection name (user needs rights to be able to change stuff)
• Subsurvey
• Component name (can not be updated.)

·        SponsorInfo   (recommend moving to the instance level in metadata)

• providerInfo  (recommend moving to the instance level in metadata)
• addInheritedComponent
• removeInheritedComponent
• abstract

·        category

·        logo (recommend moving to the instance level in the metadata)

• display (works on component level)

These changes apply ONLY to the instance listed.

·        extractionHost

·        tabulationHost

·        restriction

Note: modifying an instance does not make sense because the user can just delete an instance and insert a new one.(We did this because it is hard to figure out changing an instance in the middle of a series.)

2. update an existing item’s information (these are not breaks in series). Note: The changes will apply to all related instances of the item.  Changes will only be made to the fields that are present in this XSD.All other fields remain as they were.

Required fields:  component, collection short name, collection long name, instance, original item name (variable id attribute)

Optional fields that can be changed:

·        name (id)

·        label

·        concept

·        security

·        unit

·        type

o       type

o       iterationgroup

o       datatype

o       decimal

o       geographyIndicator

o       interval

o       isweight

o       length

o       weightvar

·        values (need to define all values for item, not just modified ones)

·        longDscr and universe?? – Because the way we store these 2 separate attributes in a single html file, how can a user update only one of the 2 parts?  Either we don’t let them update at all, or we make them define both in the update XSD.  Or we completely change the way we store them or collect them.CURRENTLY in the mifModify XSD we DO NOT allow a user to update the universe.  However, if the user updates the longDscr, they will overwrite any universe info that was contained in the original html file.

·        attachment ???

·        synonyms????

·        Notes????

Just a list of changes/rules (bh FIXME)

• high level documentation not used.  (producer, version, and abstract because do don’t support in metadata repository.)
• dataset --- refDate, releaseDate, collectDate keywords, notes not used in ferret so not in rules.

***** VARIABLES *****

Stuff not being used in repository if not being updated  (embargo)

mifDelete:  This deletes an entire component, instance, or individual items. Deletes in a roll up structure format – will delete the parent if no children exist any longer (therefore it can end up deleteing subsurveys and collections.

User defines these things for deletions: collection, subsurvey, component, instance, variable(s) – and based on what parts are defined, the deletions will be as follows:

1. If user defines ALL parts – then the variables specified will be deleted for that instance.
• If all variables for the instance were listed, then the instance will also be removed.(Can also define whether deleting the variable for the one specified instance only, or the variable for all related instances, or across all occurrences of the variable (multiple definitions)).
• If a user deletes an item, and the item continues and it was at the end of the series, then it will still leave the continue flag there until a new instance is ingested.Once the new instance has been loaded then it will put a discontinue flag on the previous instance because this would leave a gap for that variable and you cannot have a continuing item skip an instance.  The reason why we did not put a stop as soon as we deleted the item is because we wanted to get back to previous state of the variable (this could be necessary if a second delete removed all remaining items for that instance).
• If a user deletes an item and the item is in the middle of the series, then it will put a discontinue flag and delete the item.
2. If user defines collection, subsurvey, component, and instance (but no variables), then the instance (and all of its variables) will be removed.If it was the only instance in the component, then the component will also be removed.
3. If user defines collection, subsurvey, and component only, then the component (and all of its instances and variables) will be removed.  If it is the only component in the subsurvey, then the subsurvey will also be removed. If there is no subsurvey, and it is the only component in the collection, then the dataset will also be removed.This is the only way that collections and subsurveys can be deleted – by deleting all components under them.

modifyTime:  All changes that are related to time are made through this.  E.g.: discontinuing all items, discontinuing/continuing specific items, and redefining items (break in series).NOTE: all references to forward and backward are using a timeline analogy.  Forward is moving towards the most recent time, and backward is moving towards the earliest time.

1. discontinue last instance (all items) – no instance or items need to be specified, if specified they will be ignored.It will only discontinue the LAST instance.
2. discontinue first instance (all items - backwards) – not yet implemented because we currently do not have the continues backward flag.
3. discontinue specific item(s)forward - instance needs to be specified (instance specified will be the LAST instance for that item)
4. discontinue specific item(s)backward - instance needs to be specified (instance specified will be the FIRST instance for that item)

5. continue an existing stopped item (can only continue one instance at a time because instance is not a true time (just a string, e.g. wave, unknown, etc.))

·        continue forward last instance– this simply changes the continuesForward flag to Y from N (continuesForward flag is proposed name change for current stopdateflag field in timeframe table)

·        continue backward first instance– this simply changes the continuesBackward flag to Y from N (continuesBackward flag is a proposed new field in timeframe table)

·        continue from past definition (you want to “pull” the previous period’s definition of an item forward or essentially continue previous variable to a later time)

·        continue from future definition (you want to “pull” the next period’sdefinition of an item backward or essentially continue a variable back to an earlier time)

newItemInExisting:  This adds new items to components that already exist, e.g. the item was initially missing in the metadata when the component was defined.

1. Define a new item in an existing instance

Required fields:  component, collection short name, collection long name, instance

The items are defined in the exact same way as in the newComponent XSD.

NOTES:  Can you add an item to more than one instance at a time or all defined instances? or do you have to add it to one, then use the timeModify to pull it to the other instances?

Redefining an item and updating an item are two completely different things.

• Redefining an item is only applicable when you have more than one instance of a dataset and are documenting changes over time.  When you redefine an item you are discontinuing the existing definition of the item (it will still exist for the old time), and inserting a new definition of the item.  This is often referred to as a break in series, since that item cannot be compared across that time when its definition changed.

• Updating an item simply means that you want to change some part of an item’s documentation, but the item still has the same meaning.

Assumptions

• If an item is defined as character, but no length is given, then we will assign it a length of 25 by default.
• If length is defined as zero or less, then an error will be given.