National Information Exchange Model (NIEM) User Guide Volume 1 www.NIEM.gov niem_logo U.S. Department of Justice Office of Justice Programs 810 Seventh Street, NW Washington, DC 20531 The Honorable Michael B. Mukasey Attorney General The Honorable Mark R. Filip Deputy Attorney General The Honorable Kevin O’Connor Associate Attorney General The Honorable Jeffrey L. Sedgwick Acting Assistant Attorney General The Honorable Domingo S. Herraiz Director, Bureau of Justice Assistance Office of Justice Programs World Wide Web Home Page www.ojp.usdoj.gov Bureau of Justice Assistance World Wide Web Home Page www.ojp.usdoj.gov/BJA For grant and funding information, contact U.S. Department of Justice, Office of Justice Programs Funding Opportunities http://www.ojp.usdoj.gov/funding BJA full logo 20070720 This project was supported by Grant No. 2007-RG-CX-K021 awarded by the Bureau of Justice Assistance. The Bureau of Justice Assistance is a component of the Office of Justice Programs, which also includes the Bureau of Justice Statistics, the National Institute of Justice, the Office of Juvenile Justice and Delinquency Prevention, and the Office for Victims of Crime. Points of view or opinions in this document are those of the author and do not represent the official position or policies of the United States Department of Justice. Table of Contents Table of Contents ...................................................................................................................... iii List of Figures ............................................................................................................................ vi List of Tables ............................................................................................................................. ix 1 Acknowledgements ............................................................................................................. 1 2 Introduction ......................................................................................................................... 2 2.1 Typographical Conventions Used in This Document ............................................... 3 3 The Need for Information Sharing ...................................................................................... 4 3.1 Challenges to Information Sharing .......................................................................... 4 3.2 Information Sharing Architectures .......................................................................... 5 4 NIEM Overview .................................................................................................................... 7 4.1 Background .............................................................................................................. 8 4.2 The Evolution of NIEM ............................................................................................. 8 4.3 NIEM Data Model ..................................................................................................... 9 4.4 Design Criteria for NIEM ........................................................................................ 11 5 NIEM Data Model Concepts .............................................................................................. 13 5.1 An Introduction to Modeling Concepts ................................................................. 13 5.2 Expressing Object-Oriented Concepts in XML: Types and Properties .................. 14 5.3 Container Elements ................................................................................................ 16 5.4 Content Elements and Reference Elements .......................................................... 17 5.5 Associations ........................................................................................................... 19 5.6 Roles ....................................................................................................................... 25 5.7 Abstract Elements and Substitution Groups .......................................................... 28 5.8 Augmentation ........................................................................................................ 34 5.9 Metadata ................................................................................................................ 38 5.10 External Adapter Types .......................................................................................... 41 6 NIEM Data Model Content ................................................................................................ 44 6.1 Architecture of NIEM Model .................................................................................. 44 6.1.1 Relationships Between the Components................................................... 45 6.2 Namespaces ........................................................................................................... 47 6.2.1 Structures .................................................................................................. 47 6.2.2 Domains .................................................................................................... 53 6.3 Standard Code Lists ................................................................................................ 56 7 Building NIEM-Conformant Data Exchanges ..................................................................... 58 7.1 NIEM Information Exchange Package Document (IEPD) Development Lifecycle .. 59 7.2 Step 1: Scenario Planning ...................................................................................... 60 7.3 Step 2: Requirements Analysis .............................................................................. 63 7.3.1 Domain Modeling Options ........................................................................ 64 7.4 Step 3: Mapping and Modeling ............................................................................. 68 7.5 Step 4: Building and Validating ............................................................................. 70 7.5.1 Subset Schema .......................................................................................... 74 7.5.2 Extension Schema ..................................................................................... 75 7.5.3 Exchange Schema ..................................................................................... 76 7.5.4 Constraint Schema .................................................................................... 76 7.5.5 Validating IEP Schemas ............................................................................. 78 7.6 Step 5: Assembling and Documenting .................................................................. 78 7.7 Step 6: Publishing and Implementing ................................................................... 79 7.8 Data Harmonization and Refactoring .................................................................... 80 8 IEPD Artifacts ..................................................................................................................... 81 9 IEPD Metadata ................................................................................................................... 83 Appendix A: Data Model Conformance Guidelines .............................................................. 85 Introduction ....................................................................................................................... 85 Conformance Rules ........................................................................................................... 85 Assistance in Developing NIEM-Conformant Schemas ..................................................... 86 Additional Remarks About Conformance .......................................................................... 86 Grant Recipients ................................................................................................................ 86 Appendix B: NIEM Tools ....................................................................................................... 88 Introduction ....................................................................................................................... 88 Help Documentation on NIEM.gov.................................................................................... 89 Registering on NIEM.gov ................................................................................................... 89 Justice Information Exchange Model (JIEM) Tool ............................................................. 89 Universal Modeling Language (UML) Tools ....................................................................... 90 Searching and Navigating the NIEM Model ...................................................................... 91 NIEM Data Model Browser .................................................................................... 91 Alternate Model Formats ....................................................................................... 92 NIEM Wayfarer ...................................................................................................... 93 Subset Schema Generation Tool (SSGT) ................................................................. 94 Map Information Exchange ............................................................................................... 98 Creating an Exchange ............................................................................................ 99 Mapping an Exchange.......................................................................................... 101 Generating Artifacts ............................................................................................. 103 Component Mapping Template (CMT) ........................................................................... 105 Building Schema Subset .................................................................................................. 105 Generating Documents Page ............................................................................... 107 SSGT Options Page ............................................................................................... 107 XML Development Tools ................................................................................................. 109 Working With IEPDs ........................................................................................................ 109 Searching the IEPD Repository ............................................................................. 110 My IEPDs .............................................................................................................. 110 Creating a NIEM IEPD........................................................................................... 111 Uploading a NIEM IEPD ........................................................................................ 115 Generating a Code List Schema ....................................................................................... 115 Migration Assistance Tool (MAT) .................................................................................... 116 Best Practices .................................................................................................................. 119 Appendix C: NIEM Resources ............................................................................................. 120 NISS Help Desk and Knowledge Base .............................................................................. 120 NIEM Web Site ................................................................................................................ 121 IEPD Clearinghouse ......................................................................................................... 121 IEPD Clearinghouse Benefits and Features .......................................................... 121 NIEM Training .................................................................................................................. 121 NIEM Documents ............................................................................................................. 122 Appendix D: Changes in NIEM Constructs Versus GJXDM 3.0.3 Constructs ...................... 123 Associations in NIEM Versus Associations in GJXDM ...................................................... 123 Roles in NIEM Versus Roles in GJXDM............................................................................. 124 Metadata in NIEM Versus Metadata in GJXDM .............................................................. 125 Appendix E: Glossary of Terms and Acronyms................................................................... 127 Glossary of Terms ............................................................................................................ 127 Glossary of Acronyms ...................................................................................................... 140 Appendix F: NIEM 2.0 Reference Schemas ........................................................................ 145 List of Figures Figure 1: NIEM at 50,000 Feet. ................................................................................................ 9 Figure 2: NIEM Core and Domains. ........................................................................................ 10 Figure 3: XML Schema Fragment Illustrating the Definition of nc:PersonType. .................... 15 Figure 4: XML Instance Fragment Illustrating the Use of nc:PersonType. ............................. 15 Figure 5: Diagram Depicting nc:PersonType. ......................................................................... 15 Figure 6: Diagram Depicting nc:AssessmentPerson. .............................................................. 16 Figure 7: XML Schema Fragment Illustrating j:DriverLicenseDrivingIncidentAssociationType. .............................................................................................................................. 16 Figure 8: XML Schema Fragment Illustrating the Definition of nc:AssessmentType. ............ 17 Figure 9: Use of nc:PersonFullName as a Content Element in PersonNameType. ................ 17 Figure 10: XML Instance Showing the Use of Content Element. ............................................ 18 Figure 11: Use of Reference Element nc:PersonFullNameReference. ................................... 18 Figure 12: XML Instance Showing Use of a Reference Element. ........................................... 18 Figure 13: Diagram Illustrating the Definition of nc:ActivityPersonAssociationType. ........... 19 Figure 14: XML Schema Fragment Illustrating the Definition of s:ComplexObjectType and s:ReferenceType. .................................................................................................. 20 Figure 15: XML Schema Fragment Illustrating nc:ActivityType and the Element nc:ActivityReference. ............................................................................................ 21 Figure 16: XML Schema Fragment Illustrating nc:PersonType and the Element nc:PersonReference. ............................................................................................. 22 Figure 17: XML Schema Fragment Illustrating nc:AssociationType and nc:ActivityPersonAssociationType. ....................................................................... 23 Figure 18: XML Schema Fragment Illustrating j:IncidentInvestigatorAssociation. ................ 23 Figure 19: XML Instance Fragment Illustrating the Use of j:IncidentInvestigatorAssociation. .............................................................................................................................. 24 Figure 20: Example of Valid XML Instance Containing Errors. ............................................... 25 Figure 21: Example of Person Role. ....................................................................................... 25 Figure 22: j:MissingPersonType as a Role of nc:PersonType. ................................................ 25 Figure 23: XML Schema Fragment Illustrating j:MissingPersonType. .................................... 26 Figure 24: XML Schema Fragment Illustrating j:MissingPersonType. .................................... 27 Figure 25: XML Schema Fragment Illustrating j:MissingPersonType. .................................... 27 Figure 26: XML Instance Fragment Illustrating the Use of j:MissingPersonType. ................. 28 Figure 27: Valid XML Instance Fragment With Error. ............................................................ 28 Figure 28: Abstract Type-Less nc:LocationState Element. ..................................................... 29 Figure 29: XML Schema Fragment Illustrating s:SimpleObjectAttributeGroup. .................... 30 Figure 30: XML Schema Fragment Illustrating nc:LocationState. .......................................... 31 Figure 31: XML Schema Fragment Illustrating can:CanadianProvinceCodeType. ................. 32 Figure 32: XML Schema Fragment Illustrating fips_5-2:USStateCodeType. .......................... 33 Figure 33: XML Instance Fragment Illustrating the Use of nc:LocationStateCanadianProvinceCode. ............................................................. 33 Figure 34: XML Instance Fragment Illustrating the Use of nc:LocationStateFIPS5- 2AlphaCode. .......................................................................................................... 34 Figure 35: Use of j:PersonAugmentationType. ...................................................................... 34 Figure 36: XML Schema Fragment Illustrating s:AugmentationType. ................................... 35 Figure 37: XML Schema Fragment Illustrating nc:PersonType. ............................................. 36 Figure 38: XML Schema Fragment Illustrating j:PersonAugmentationType. ......................... 36 Figure 39: XML Schema Fragment Illustrating ext:PersonType. ............................................ 37 Figure 40: XML Instance Fragment Illustrating the Use of ext:Person. ................................. 37 Figure 41: XML Schema Fragment Illustrating s:MetadataType. ........................................... 39 Figure 42: XML Schema Fragment Illustrating j:JusticeMetadataType.................................. 40 Figure 43: XML Instance Fragment Illustrating the Use of j:JusticeMetadata ........................ 40 Figure 44: Use of j:JusticeMetadataType. .............................................................................. 40 Figure 45: Use of External Adapter Type geo:SingleSite LandmarkAddressType. ................. 42 Figure 46: Definition of External Type addr:MunicipalJurisdiction and addr:USPSPlaceName. .............................................................................................................................. 42 Figure 47: XML Instance Showing the Use of an External Adapter Type. .............................. 43 Figure 48: Use of Elements From Structures in Reference Schema Documents. .................. 46 Figure 49: NIEM Components. ............................................................................................... 47 Figure 50: Definition of nc:PersonReference in NIEM. .......................................................... 48 Figure 51: Definition of the s:ReferenceTypeNIEM Core. ...................................................... 48 Figure 52: IEPD Lifecycle. ....................................................................................................... 59 Figure 53: An Example of a Domain Modeling Spreadsheet. ................................................ 65 Figure 54: Informal Graphical Model. .................................................................................... 66 Figure 55: Extract From the Amber Alert Domain Model. ..................................................... 67 Figure 56: Example of an Amber Alert Mapping Document. ................................................. 69 Figure 57: Example of Schema Development Process. ........................................................... 73 Figure 58: The Tools Help Feature Has Been Greatly Enhanced With NIEM 2.0. .................. 89 Figure 59: Data Model Browser. ............................................................................................ 92 Figure 60: NIEM Wayfarer Tool With Example Graphical Output. ......................................... 94 Figure 61: The SSGT Is the Primary Entry Point Into Building NIEM Subset Schemas. .......... 95 Figure 62: The SSGT Includes a Number of User-Selectable Options to Help Refine Your Search.................................................................................................................... 97 Figure 63: Create an Exchange. .............................................................................................. 99 Figure 64: Change the Default Exchange Name..................................................................... 99 Figure 65: Uploading Data Model to Exchange. .................................................................. 100 Figure 66: Process to Begin Mapping Exchange Elements to NIEM. ................................... 100 Figure 67: Find Equivalent NIEM Terms. .............................................................................. 101 Figure 68: More Information Regarding Selected Components. ......................................... 101 Figure 69: Search for Additional Matches............................................................................ 102 Figure 70: Comments and Mapping Categories. .................................................................. 103 Figure 71: Generate Artifacts. .............................................................................................. 104 Figure 72: Select Component for Subset. ............................................................................ 106 Figure 73: Generate Documents Page. ................................................................................ 107 Figure 74: Load a Wantlist or Change Release Versions on the SSGT Options Page. .......... 108 Figure 75: Search, Edit, Create, or Upload an IEPD From the Overview Screen. ................. 110 Figure 76: Review and Edit Artifacts. ................................................................................... 110 Figure 77: Code List Template. ............................................................................................. 116 Figure 78: Code List Generation Tool. .................................................................................. 116 Figure 79: Select a Wantlist File and Version to Migrate. .................................................... 117 Figure 80: Migration Summary. ........................................................................................... 118 Figure 81: Example of GJXDM Property Represented as a Content Element. ..................... 123 Figure 82: Example of GJXDM Property Represented as a Reference Element. ................. 123 Figure 83: GJXDM XML Schema Fragment Illustrating the Definition of j:ActivityType. ..... 124 Figure 84: Definition of j:MissingPersonType. ..................................................................... 124 Figure 85: GJXDM XML Schema Fragment Illustrating the Definition of j:MissingPersonType. ............................................................................................................................ 125 Figure 86: XML Schema Fragment Illustrating the Definition of j:SuperType in GJXDM. .... 126 List of Tables Table 1: About This Document. ................................................................................................ 3 Table 2: Comparison of Terminology in the NIEM Data Model, XML, and UML. .................. 14 Table 3: Content Definition Namespace. ............................................................................... 47 Table 4: NIEM Core Namespace............................................................................................. 49 Table 5: Emergency Management Domain. ........................................................................... 53 Table 6: Immigration Domain. ............................................................................................... 53 Table 7: Intelligence Domain. ................................................................................................ 54 Table 8: Infrastructure Protection Domain. ........................................................................... 54 Table 9: International Trade Domain. .................................................................................... 55 Table 10: Justice Domain. ...................................................................................................... 55 Table 11: People-Screening Domain. ..................................................................................... 56 Table 12: Scenario Planning Tasks. ........................................................................................ 63 Table 13: Requirements Analysis Tasks. ................................................................................ 68 Table 14: Map and Model Tasks. ............................................................................................ 70 Table 15: Build and Validate Tasks. ......................................................................................... 78 Table 16: Assemble and Document Tasks. .............................................................................. 79 Table 17: Publish and Implement Steps.................................................................................. 80 Table 18: Data Harmonization and Promotion. ...................................................................... 80 Table 19: IEPD Artifacts. ......................................................................................................... 82 Table 20: IEPD Metadata. ...................................................................................................... 84 Table 21: List of NIEM Tools. .................................................................................................. 88 1 Acknowledgements 1 The Bureau of Justice Assistance (BJA) and the NIEM Project Management Office (PMO) would like to thank the many individuals who contributed to the development of this document. This User Guide would not have been possible without the tremendous amount of work and assistance of all the authors, editors, and volunteer reviewers. BJA and the NIEM PMO are especially grateful to members of the following committees and organizations for their collaborative efforts: . Georgia Tech Research Institute (GTRI) . Global Justice Information Sharing Initiative (Global), XML Structure Task Force . IJIS Institute XML Advisory Committee . National Center for State Courts (NCSC) . NIEM PMO Committee Members and Volunteers . SEARCH, The National Consortium for Justice Information and Statistics . U.S. Department of Homeland Security (DHS) NIEM User Guide Development Team Mr. Philip Ardire Tetrus Consulting Ms. Carrie Boyle Bearingpoint Ms. Andrea Cafarelle Tetrus Consulting Mr. Tom Carlson Tom Carlson Consulting Mr. Scott Chontow Bearingpoint Mr. Donald Gabbin IJIS Institute Mr. Ashwini Jarral IJIS Institute Mr. Chandra Jonelagadda Tetrus Consulting Mr. Vivek Misra URL Integration Ms. Lisa Neal IJIS Institute Mr. Prem Neelakanta Analyst International Mr. Vipul Patel IJIS Institute Ms. Catherine Plummer Information Sharing LLC Mr. Sharad Rao Tetrus Consulting Mr. Dave Usery URL Integration 2 2 Introduction 3 The National Information Exchange Model (NIEM) is a partnership of the U.S. Department 4 of Justice (DOJ) and the U.S. Department of Homeland Security (DHS). It is designed to develop, 5 disseminate, and support enterprise-wide information sharing standards and processes, 6 providing a framework for communities of interest throughout the nation to collaborate and 7 share critical information effectively. NIEM enables information sharing across all levels of 8 government, including Federal, state, local, and Tribal governments, and is supportive of both 9 day-to-day operations and real-time emergency situations.1 10 The NIEM User Guide Volume I provides detailed guidance about how to develop 11 information exchanges utilizing this model. It provides a detailed description of the rationale for 12 the creation of NIEM, an architectural overview, and technical concepts derived from NIEM 13 Program Management Organization (PMO) documentation. This volume takes the reader 14 further into a methodology for defining the business requirements of the information exchange, 15 as well as creating an Information Exchange Package Documentation (IEPD) that fully specifies 16 the exchange in conformance with NIEM guidelines. Also included is information about tools to 17 assist development, resources for education and peer assistance, emerging technologies and 18 how they relate to NIEM, and the national partners that bring it all together. 19 The primary audience for this document is engineers and developers who intend to use the 20 NIEM standard to support interagency information sharing. The reader is expected to have an 21 understanding of the concepts of object oriented design, UML, and XML technologies. 22 This NIEM User Guide consists of the following sections: 23 1 http://www.niem.gov/whatIsNiem.php. Section Description 1 Acknowledgements This section lists the individuals and agencies that were involved in the creation of the NIEM User Guide. 2 Introduction This section provides an overview of the document and describes the notations used throughout. 3 The Need for Information Sharing This section provides an overview of the need for information sharing, some key concepts required for information sharing and an overview of NIEM. 4 NIEM Overview This section provides an overview of the NIEM data model. 5 NIEM Data Model Concepts NIEM Data Model Concepts This section discusses data model concepts of the NIEM model. It details types, properties, namespaces, and other concepts in NIEM. 6 NIEM Data Model Content This section describes the content of the NIEM data model. It identifies the current domains that the NIEM data model covers. 7 Building NIEM-Conformant Data Exchanges This section discusses the suggested methodology for building NIEM-conformant data exchanges. 8 IEPD Artifacts This section identifies the artifacts that may be produced as a result of an IEPD development. 9 IEPD Metadata This section defines the metadata that must be created to enable this IEPD to be discovered by other individuals when they search the IEPD repository. Section Description Appendix A: Data Model Conformance Guidelines This appendix discusses the conformance guidelines for NIEM. Appendix B: NIEM Tools This appendix discusses the tools available to the reader to develop NIEM-conformant IEPDs. Appendix C: NIEM Resources This appendix discusses the resources that are available to the reader for obtaining additional information about NIEM. Appendix D: NIEM Constructs vs. GJXDM Constructus This appendix briefly demonstrates some differences between the NIEM and GJXDM constructs. Appendix E: Glossary of Terms and Acronyms This appendix provides definitions for terms and acronyms that appear in bold throughout this document. Appendix F: NIEM 2.0 Reference Schemas This appendix presents the code lists and external schemas that are utilized in NIEM. Table 1: About This Document. 24 2.1 Typographical Conventions Used in This Document 25 Throughout this document, the following typographical conventions provide you with clues 26 as to the significance or context of the material being discussed. 27 . This is an alert. When you see information presented in this manner, pay special 28 attention—information presented in this manner is critical to your understanding of 29 the concept being discussed. 30 31 . This is a note. Information presented in this manner is important but not critical to your 32 understanding of the concept being discussed. 33 34 35 Example code appears in this typeface. 36 37 38 3 The Need for Information Sharing 39 Information sharing involves the business processes, policies, procedures, architecture, and 40 governance that support effective decision-making and mission-focused actions by providing 41 timely, accurate, and relevant information to the appropriate individuals across all levels of 42 government. Essentially, it is this need that makes the business case for the creation and use of 43 a standard such as NIEM. 44 A variety of emergency situations in recent years have demonstrated the potentially tragic 45 consequences that can result from the inability of jurisdictions and agencies to effectively share 46 information. Terrorist attacks, natural disasters, and large-scale organized criminal incidents 47 serve as case studies that reveal weaknesses in our nation’s information sharing capabilities. 48 Moreover, enterprise-wide information sharing is also required to support the critical day-to-49 day operations of federal, state, local, and tribal officials. 50 Current information collection and dissemination practices have not been planned as part 51 of a unified national strategy but, rather, have evolved incrementally over time to meet specific 52 one-off challenges as they have surfaced. Agencies are often unable to effectively share 53 information in a timely, secure manner, and there can be fundamental differences in the nature 54 and understanding of information that can be shared between agencies. While sharing does 55 occur today, it often occurs to a limited degree, or within stovepipe information systems. A 56 tremendous quantity of information that should be shared is still not effectively done, nor is this 57 information utilized effectively among relevant communities of interest (COIs). 58 3.1 Challenges to Information Sharing 59 Previous efforts to improve this situation have been beset by a multitude of challenges. 60 These challenges include: 61 . Stovepipe information systems leading to inability to connect the dots. 62 Independent agencies have separate data systems, funding streams, and chains 63 of command. This separation of data and ownership can obscure relationships 64 and inhibit the ability of law enforcement, justice and public safety, and 65 homeland security officials to have the right information at the right time to 66 assist in proper decision making. By providing these leaders with the 67 technology framework to share information, the nation’s capacity to combat 68 crime and terrorism, as well as improve the administration of justice and 69 homeland security, can be greatly improved. 70 . Large number of organizations at the Federal, State, Local and Tribal levels 71 including the private section. There are a large number of jurisdictions at the 72 public level as well the private level with disparate information systems, 73 governance and activities that need to share information. The sheer number 74 of organizations and their autonomous nature engender inconsistent policies, 75 practices, and systems, thereby making coordination more difficult. 76 . Lack of consistent policies and practices. Information sharing practices and 77 policies often vary from agency to agency with respect to such issues as privacy 78 protection, security, data quality control, and access. These inconsistent 79 approaches combined with lack of advised memoranda of understanding 80 (MOUs) in place make it difficult—and sometimes illegal—to share information 81 with other agencies. 82 . Lack of common standards for the description and definition of data and 83 information. Without common standards, data is developed and used within 84 information systems in a myriad of different ways, causing data duplication, 85 increasing inaccuracies, and making information usage and alignment across 86 jurisdictions very difficult. In addition, the consistent definition of the 87 sensitivity level or classification of data is often lacking across potential 88 partners, inhibiting confidence in the sharing of secure and protected 89 information. 90 . Interagency mistrust. As a result of inconsistent policies and practices, those 91 who do share sensitive information cannot always be sure how it will be used, 92 whether it will be protected, how it will be disseminated to a third party, and 93 who will ultimately have access to it. 94 . Categorization of otherwise shareable information into non-shareable 95 categories. Another barrier to information sharing is created when 96 information that should be categorized as shareable is categorized in a way 97 that prevents it from being shared. This is primarily due to the lack of 98 department-wide training and awareness strategy with regard to information 99 handling. 100 . Privacy with regard to information sharing. Ethical and legal obligations 101 compel every professional in the justice system to protect privacy interests 102 when sharing justice information. Today, increased security needs not only 103 dictate enhanced justice information sharing but also highlight the need to 104 balance privacy protection and justice information access. The ease of digital 105 access now makes analysis of privacy obligations a more complex process. 106 Nonetheless, the underlying foundations for privacy policy exist in our current 107 laws and customs. Constitutions, statutes, regulations, policies, procedures, 108 and common law requirements still control justice entity collection and sharing 109 of information. What is new is the need for justice practitioners to articulate 110 the rules that control their information gathering and sharing activities in a 111 manner that both supports information sharing and protects constitutional 112 privacy rights. 113 . Lack of coordination on information sharing efforts. In many cases, regional 114 information sharing initiatives have not been coordinated with one another or 115 with their federal partners and vice versa. Since the terrorist attacks of 116 September 11, 2001, the President and Congress have sought to address these 117 challenges by mandating information sharing through various Executive Orders 118 and by directing agencies to increase cooperation and sharing, especially as it 119 relates to critical information that affects the security of the homeland. 120 3.2 Information Sharing Architectures 121 The Information sharing architectures that have been developed provide the framework 122 for coordinating business processes, information exchanges, technology components, and 123 performance metrics in relation to information sharing. These include the Federal Enterprise 124 Architecture (FEA), the Justice Reference Architecture (JRA) developed by the Global 125 Infrastructure and Standards Working Group (GISWG), the ISE Enterprise Architecture 126 Framework (EAF) developed by the Program Manager for the Information Sharing Environment 127 (PM-ISE). 128 These architectures support the sharing of information. NIEM is not a competitor to those 129 activities, but rather complements them as a method used to implement the data exchange 130 layer within these architectures. 131 4 NIEM Overview 132 NIEM, as a platform for information sharing, is based on eXtensible Markup Language 133 (XML). XML is a structured language for describing information being sent electronically by one 134 entity to another. XML schema defines the rules and constraints for the characteristics of the 135 data, such as structure, relationships, allowable values, and data types. 136 XML is: 137 . In-text format, readable by both machines and humans 138 . license-free 139 . platform-independent 140 . well-supported by industry 141 XML specifications2 are guided by the W3C standards. 142 The NIEM data model is represented in XML but provides specialized XML tag names and 143 other structure for data that is constrained to meet the specific information exchange 144 requirements of the justice and homeland security domains. In other words, NIEM utilizes XML 145 to provide a concise and defined vocabulary for sharing critical information throughout the 146 nation. This is true regardless of whether the agency sharing the information is local, state, 147 tribal, or federal and regardless of whether the information is exchanged horizontally or 148 vertically within existing or emerging systems. 149 2 http://www.w3.org/XML/. . NIEM provides a common language with which federal, state, local, and tribal agencies 150 can describe, structure, and share critical information in both emergency and routine 151 situations. NIEM is designed to facilitate information exchange among different 152 domains, such as justice, public safety, emergency and disaster management, 153 intelligence, and homeland security. NIEM makes this possible by providing the data 154 standards and exchange development methods for defining these cross-domain 155 exchanges. 156 157 158 4.1 Background 159 DOJ and DHS launched the NIEM program on February 28, 2005. Among other 160 requirements, NIEM complies with Homeland Security Presidential Directive-5 (HSPD-5),3 which 161 assigns the Secretary of Homeland Security the role of principal federal official for domestic 162 incident management. The Homeland Security Act of 20024 charges the Secretary with the 163 responsibility for coordinating federal operations within the United States to prepare for, 164 respond to, and recover from terrorist attacks, major disasters, and other emergencies. The 165 Intelligence Reform and Terrorism Prevention Act of 2004 (IRTPA)5 was signed into law in 166 December 2004, and in 2005, Executive Order 133886 was issued by the President. These acts 167 and the administrative direction require U.S. government organizations to strengthen the 168 sharing of terrorism information between organizations and appropriate authorities of local and 169 state governments and protect the ability of organizations to acquire this additional 170 information. 171 3 http://www.fas.org/irp/offdocs/nspd/hspd-5.html. 4 http://www.dhs.gov/xlibrary/assets/hr_5005_enr.pdf. 5 http://travel.state.gov/pdf/irtpa2004.pdf. 6 http://www.fas.org/irp/offdocs/eo/eo-13388.htm. 4.2 The Evolution of NIEM 172 In the late 1990s, the state and local criminal justice community began to focus on sharing 173 information rapidly and effectively to serve a variety of public safety needs. The advent of XML 174 provided the technology with which information could be exchanged more efficiently and cost 175 effectively. The Global Justice XML Data Model (GJXDM) vocabulary was derived from user 176 requirements and was driven from the “bottom up” by active practitioners in the justice and 177 public safety fields. The unique development approach taken with GJXDM provided an 178 opportunity for national organizations to assist and support the process of sharing critical justice 179 information where that information originates—at the state, local, and tribal levels. 180 GJXDM demonstrated the value of information sharing and helped promote the business 181 case for NIEM, which now extends that concept on a national level. NIEM includes not only the 182 Justice (JXDM) domain but also represents others, such as intelligence, emergency 183 management, immigration, infrastructure protection, international trade, and screening. NIEM 184 actively encourages federal agency participation while continuing to support state and local 185 requirements and interoperability standards. NIEM provides component-based resources that 186 are reusable and portable to any organization or platform. 187 Today, the stated objectives of the NIEM PMO are to: 188 . Bring stakeholders together to identify information sharing requirements for 189 operational and emergency situations. 190 . Maintain a National Data Model and Reference Vocabulary containing common 191 and domain-specific data components that pertain to agency information 192 needs to facilitate development of discrete information exchanges. 193 . Develop standards, a common vocabulary, and an online repository of 194 exchange standards to support information sharing. 195 Developing and implementing NIEM-based exchanges allows agencies to leverage existing 196 investments in information systems by building the bridges to connect them. NIEM standards 197 enable different information systems to share and exchange information, irrespective of the 198 particular technologies in use in those information systems. Moreover, creating and adopting 199 NIEM standards means that local, state, tribal, and federal organizations can reap significant 200 cost benefits through adoption and reuse, rather than building proprietary, single-use software 201 from scratch. The fact that NIEM requirements are driven from the user community rather than 202 a Federal mandate paves the way for faster adoption, and more closely aligned outcomes 203 between the NIEM PMO and its constituents. 204 4.3 NIEM Data Model 205 The NIEM data model provides the reference vocabulary for consistent and reusable intra- 206 and interdomain information exchanges. The structure and meaning of NIEM data are defined 207 by the model and dictionary and are represented as XML schema, thereby providing a common 208 framework for information exchange. As part of the NIEM 2.0 release, the model can also be 209 viewed as a spreadsheet7 or in a database format. 210 The fundamental building block of NIEM is a data component. Data components are the 211 basic business data items that describe common concepts used in general business activities. 212 213 7 http://www.niem.gov/topicIndex.php?topic=spreadsheet. Figure 1: NIEM at 50,000 Feet. 214 Figure 1 illustrates that NIEM is modeled to be able to describe people, places, things, and 215 events and the relationships between all of them at different points in time. 216 By far, activity makes up the bulk of the model, with person information coming in second. 217 While each of these categories represents a stand-alone entity, each is structured such that it 218 can also be associated with other categories. 219 220 The NIEM architecture consists of two sets of vocabularies—NIEM Core and the individual 221 NIEM domains. NIEM Core includes Universal (U) and Common (C) components. The identities 222 for U and C components in NIEM Core are maintained with metadata. Universal data 223 components are concepts that are commonly understood across all business domains, such as 224 dates, times, and locations. They do not have to appear in every exchange and do not have to 225 apply all the time—they simply have to be well-defined and well-known enough to be 226 understood by all (or the majority of) domains. Common data components, on the other hand, 227 are used in exchanges between two or more domains but not universally shared. 228 By contrast, the individual NIEM domains contain domain-specific data components. As 229 illustrated in Figure 2, the domains of Emergency Management, Justice, Infrastructure 230 Protection, Intelligence, International Trade, and Immigration are currently participating in 231 NIEM. Additional domains will be added as policy evolves and operational requirements 232 emerge. 233 234 235 236 Domain-Specific Items with semantic meaning relevant primarily to their own domain. NIEM Core and Structures Items that exist and have same semantic meaning across all or several domains. Figure 2: NIEM Core and Domains. 237 As of version 2.0, NIEM consists of 3,985 data elements and 777 data types. The elements 238 are grouped into namespaces—NIEM Core or one of the seven domains. 239 These core components are commonly understood and their meanings are agreed to by 240 many, if not all, domains. The standardization of these core components provides significant 241 potential for increased interoperability among and between justice and public safety 242 information systems. Standardization in this manner provides each of us with functionally 243 equivalent or interchangeable components of the system or process in which they are used, 244 regardless of our individual system differences. 245 The data model and dictionary are combined into one database—a component 246 repository—which allows the consistent generation of several products that can be consumed 247 by the sharing community: 248 . The NIEM schema 249 . Numerous external code table schemas 250 . A NIEM documentation spreadsheet 251 It is recommended that new users acquaint themselves with the NIEM Component 252 Mapping Tool (CMT) spreadsheet,8 which is provided as a Microsoft Excel file for easy 253 navigation. The NIEM CMT spreadsheet provides all the element names organized hierarchically 254 under the domains (NIEM Core, Emergency Management, Justice, etc.) with hyperlinks to 255 related elements. The spreadsheet also provides information as to the type of data being 256 represented (date, integer, Boolean, string, etc.) and a precise definition of each dictionary 257 component. The definitions represent a commitment to provide reusable components that 258 mean the same thing to all domains. 259 8 http://www.niem.gov/topicIndex.php?topic=spreadsheet. 9 http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_detail.htm?csnumber=1758. 4.4 Design Criteria for NIEM 260 The primary goal for NIEM has been to develop a common set of reusable, extensible XML 261 data components that could be combined in documents, transactions, and messages that are 262 consistently structured to support interoperability between systems. The following design 263 criteria were used in the development of NIEM: 264 . NIEM should be constructed from actual functional requirements, reference 265 documents, use cases, and business-context components. 266 . An object-oriented data model, named types, and extensions are best suited to 267 the goals of interagency information exchange. 268 . The composition of the data dictionary should be over-inclusive and optional to 269 allow users to pick and choose appropriate building blocks for their data 270 exchanges. 271 . NIEM element and attribute tag names should be based on relevant 272 international standards for electronic data exchange, especially ISO/IEC 11179-273 5:1995—Specification and Standardization of Data Elements9, as discussed in 274 the NIEM Naming and Design Rules (NDR). Additional source standards 275 include, but are not limited to: 276 . W3C XML Schema Specification and RDF and RDF Schema Specification.10 277 . The Federal CIO Council Draft Federal XML Schema Developer’s Guide.11 278 . UN/CEFACT ebXML Core Components Technical Specification 2.01.12 279 . Dublin Core Metadata for Documents.13 280 . U.S. Department of Defense 5015.02-STD Design Criteria Standard for 281 E-RMS Applications.14 282 . The OASIS XML Common Biometrics Format.15 283 . The ASC X12 Reference Model for XML Design.16 284 . NIEM continues to evolve, so the data model must facilitate change and 285 extension as required. 286 . Extension methods should comply with NIEM Naming and Design Rules 287 (NDR)17 to minimize the impact on prior schema and code investments by 288 practitioners and developers. 289 . NIEM must provide migration paths for evolution to new technologies, such as 290 Resource Description Framework (RDF) and Web Ontology Language (OWL).18 291 10 http://www.w3.org/XML/Schema#dev. 11 http://www.xml.gov/documents/in_progress/developersguide.pdf. 12 http://www.unece.org/cefact/ebxml/CCTS_V2-01_Final.pdf. 13 http://dublincore.org/documents/. 14 http://www.dtic.mil/whs/directives/corres/pdf/501502std.pdf. 15 http://www.oasis-open.org/committees/download.php/3353/oasis-200305-xcbf-specification-1.1.doc. 16 http://www.x12.org/x12org/xmldesign/X12Reference_Model_For_XML_Design.pdf. 17 http://niem.gov/topicIndex.php?topic=file-NDR-withoutLineNum. 18 RDF and OWL are semantic Web standards that provide a framework for asset management, enterprise integration, and the sharing and reuse of data on the Web. NIEM provides a mechanism through which standards for information exchange can be 292 defined with a high degree of granularity. 293 5 NIEM Data Model Concepts 294 5.1 An Introduction to Modeling Concepts 295 NIEM is a standardized data model and a reference vocabulary implemented in XML 296 schema. The NIEM data model states exactly and explicitly the meaning of a given concept or 297 relationship. Accordingly, an XML instance that conforms to the NIEM XML schema also has 298 specific meaning. The purpose of NIEM is to provide a standard—but extensible—format for 299 use in the exchange of information between information systems. 300 NIEM employs several constructs that address common concerns in the design of data 301 models that represent information being exchanged between software systems. 302 . Types and Properties: Representations of the physical and conceptual things 303 being communicated. 304 . Container Elements: Elements whose presence in types represents 305 semantically weak relationships. 306 . Content Elements and Reference Elements: Two semantically equivalent ways 307 to represent the properties of a type. 308 . Associations: Representations of the relationships that a type 309 (e.g., “PersonType”) has with other types (e.g., “VehicleType,” “ActivityType”) 310 that do not create duplicate copies of the type in question (“PersonType”). 311 . Roles: Representations of the different roles (e.g., “VictimType,” 312 “WitnessType”) that a type (e.g., “PersonType”) plays in its relationships with 313 other types (e.g., “IncidentType,” “CaseType”) that do not create multiple, and 314 possibly conflicting, specializations of the type in question (“PersonType”). 315 . Code Lists: Generic representations of enumerated code values of a type. 316 . Augmentation: Representation of a reusable bundle of properties 317 (e.g., “PersonAugmentationType” containing properties “DriverLicense,” 318 “PersonFootPrint,” etc.) for the purpose of augmenting the definition of an 319 existing type (e.g., “PersonType”) that does not create multiple, and possibly 320 conflicting, specializations of the type in question (“PersonType”). 321 . Metadata: Representation of metadata of types in a flexible and extensible 322 manner. 323 . External Adapter Types: Usage of non-NIEM types in a NIEM-conformant 324 schema. 325 Each of the above-mentioned constructs comes with a prescribed mechanism to follow 326 when designing NIEM-conformant XML schema types and when using elements of those types 327 in XML instances. This chapter describes and exemplifies these constructs and mechanisms. 328 329 5.2 Expressing Object-Oriented Concepts in XML: Types and Properties 330 The NIEM data model consists of “types” (of things) that have “properties” and that 331 participate in “relationships” with other “types” (of things). 332 A type is a description of a set of things that share the same properties, relationships, and 333 semantics. For example in NIEM, “PersonType” and “VehicleType” represent persons and 334 vehicles—kinds of things. 335 A property is a named characteristic of a type. For example, “PersonBirthDate” is a 336 property of “PersonType.” Furthermore, the property is of a specific type itself. For example, 337 ”PersonBirthDate” is itself of type “DateType.” 338 A relationship may be modeled as either a type or a property. For example in NIEM, a 339 relationship between persons and vehicles is represented by the type 340 “PersonVehicleAssociationType.” 341 An object is an instance of a type and is an abstraction of a specific physical thing or a 342 conceptual thing. Also, in an object, the properties have values. For example, John Smith, a 343 specific person, would be an object of type “PersonType” with the property “PersonBirthDate.” 344 Also, for John Smith, the property “PersonBirthDate” may have a value of “1970-01-01.” 345 An object may have a unique ID within an XML instance, but it is not required to have a 346 globally unique identifier. The presence of specific objects in an exchange makes the assertions 347 that: 348 . Objects exist. 349 . Objects have properties. 350 . Objects participate in relationships. 351 The NIEM data model is explicit, not implicit. If the data says a person’s name is John Smith, 352 it is not implying that he does not have other names or that John Smith is his legal name or that 353 he is different from a person known as Bob Jones. The only assertion being made is that one of 354 the names by which this person is known is John Smith. 355 As shown in Table 2, types, properties, and objects in the NIEM data model have equivalent 356 concepts in XML Schema and Unified Modeling Language (UML). 357 NIEM Data Model XML Schema/XML Instance UML Type e.g., “PersonType” Complex Type or Simple Type e.g., nc:PersonType Class Property e.g., “PersonBirthDate” of type “DateType” Element or Attribute e.g., nc:PersonBirthDate of type nc:DateType Attribute Object e.g., “Person” Element or Attribute e.g., nc:Person Instance/Object Table 2: Comparison of Terminology in the NIEM Data Model, XML, and UML. 358 359 In XML schema, a type is represented by a Simple Type or a Complex Type. A property is 360 represented by an attribute or an element. An object is represented by an element in an XML 361 instance fragment that conforms to the Simple Type or the Complex Type definition. 362 Consider the following fragment from the NIEM XML schema. The XML schema type 363 nc:PersonType represents the NIEM Data Model type “PersonType.” The element 364 nc:PersonBirthDate represents the property “PersonBirthDate.” Finally, the element 365 nc:AssessmentPerson of nc:PersonType represents an object of “PersonType.” 366 367 368 369 370 371 … 372 373 374 375 … 376 377 … 378 379 380 381 382 383 384 385 Figure 3: XML Schema Fragment Illustrating the Definition of nc:PersonType. 386 Next, consider the fragment below, which shows an XML instance containing 387 nc:AssessmentPerson, where the element nc:PersonBirthDate has a value of “1970-01-01.” 388 389 390 391 … 392 1970-01-01 393 … 394 395 396 Figure 4: XML Instance Fragment Illustrating the Use of nc:PersonType. 397 In UML, a NIEM Data Model type can be represented by a UML class, a NIEM Data Model 398 property by a UML attribute, and a NIEM Data Model object by a UML instance. For example, 399 the NIEM Data Model type “PersonType” can be depicted as follows: 400 401 . . . nc:PersonBirthDate. . . nc:PersonType Figure 5: Diagram Depicting nc:PersonType. 402 403 In another example, the NIEM data model object “AssessmentPerson” containing the 404 property “PersonBirthDate” with a value of “1970-01-01” could be depicted as in Figure 6. 405 406 . . . nc:PersonBirthDate = 1970-01-01. . . nc:AssessmentPerson : nc:PersonType Figure 6: Diagram Depicting nc:AssessmentPerson. 407 5.3 Container Elements 408 There are two levels of semantics that can be associated with the presence of an element 409 in a type—weak semantics and strong semantics. Consider for example, 410 j:DriverLicenseDrivingIncidentAssociationType, which represents an association between a 411 driver’s license and a driving incident and contains an element nc:Person of nc:PersonType. The 412 presence of the nc:Person element does not establish what kind of relationship exists between 413 j:DriverLicenseDrivingIncidentAssociationType and nc:PersonType, only that there is a 414 relationship. This is an example of a semantically weak relationship. In such a case, the element 415 nc:Person is called a “container element” because it only serves the purpose of containing an 416 object of nc:PersonType, while leaving the exact meaning unstated. 417 418 419 420 421 422 424 425 426 427 428 429 430 431 432 … 433 434 435 436 437 438 Figure 7: XML Schema Fragment Illustrating j:DriverLicenseDrivingIncidentAssociationType. 439 If you contrast this situation with that of nc:AssessmentType, which represents an 440 evaluation, appraisal, or assessment of something or someone and contains the element 441 nc:AssessmentPerson of nc:PersonType, it is clear that the person referenced by the element 442 nc:AssessentPerson was responsible for an assessment of some type, relevant to the exchange 443 being modeled. The more descriptive name, nc:AssessmentPerson, makes the relationship 444 between it and nc:AssessmentType a semantically strong relationship. 445 446 447 448 449 450 451 452 453 454 … 455 456 457 458 459 460 461 Figure 8: XML Schema Fragment Illustrating the Definition of nc:AssessmentType. 462 Note that the concept of “container element” is only notional. There are no formalized 463 rules about what makes an element a container element. The distinction, however, between 464 container and noncontainer elements is still useful in identifying the meaning that can be 465 explicitly associated with the presence of the element in a type. 466 . One caveat when working with NIEM—When looking for something, do not forget to 467 look upward through all the parent elements for inherited properties. 468 5.4 Content Elements and Reference Elements 469 There are two forms in which an element may be present in a type—as a content element 470 or as a reference element. A content element occurs in the definition of its containing type. For 471 example, nc:PersonFullName element occurs as a content element in its containing element 472 nc:PersonNameType in the following XML schema fragment. 473 474 475