Table of Contents
Executive Summary 1
Conclusions 3
Recommendations 4
Integrated Cross-Disciplinary Research Programs 5
Independent Research Thrusts 5
Theory Development 6
Uncertainty, Dynamic Adaptability, and Rational Behavior 6
Data Collection Methods 7
Federated Models 7
Validation and Usefulness 8
Tools and Infrastructure for Model Building 9
Multidisciplinary Conferences and Workshops 9
Roadmap for Future Research and Development 10
Background and Need for Organizational Models 11
Introduction 13
Study Task and Objectives 14
National Academies' Response 15
The Committee's Approach 15
Defining the Project Scope 16
Gathering Data 16
Data Analysis and Review 16
Concepts and Definitions 16
Cautions for IOS Modeling 19
Organization of the Report 20
References 22
Military Missions and How IOS Models Can Help 23
Military Missions Now and into the Future 24
Overarching Strategy and Operational Enablers 24
Dimensions of the New Battlespace 26
The Impact of Urbanization 26
The Growing Importance of Pre- and Postconflict Operations 28
Changes in the Nature and Scale of Intervention Operations 30
How IOS Behavioral Models Can Help the Military 32
Potential Use of IOS Models for Analysis, Forecasting, and Planning 34
Models for Understanding, Forecasting, Shaping, and Responding to Adversary Behavior 36
Models for Understanding, Forecasting, and Shaping Societal Behavior 38
Models for Understanding Enemy Command and Control Structures 39
Models for Training and Mission Rehearsal 40
Models for Military Systems Development, Evaluation, and Acquisition 42
Models for Enabling Command and Control Weapons Systems 43
Representative Model-Addressable Problems in a Scenario Context 45
Overview of Current DoD IOS Modeling Efforts 48
The DMSO Master Plan for Modeling and Simulation 48
Selected Current DoD Behavioral Modeling Efforts 51
OneSAF Family of Models and Simulations 52
Task Network Models and Tools 52
Cognitive and Cognitive-Affective Architectures and Models 53
Multiagent Systems 54
Massively Multiplayer Online Gaming 54
DIME/PMESII Models 55
Simulation Frameworks and Tools 58
Other Efforts 58
Major Challenges for Development of IOS Models for Military Applications 58
Interoperability Challenges 59
Data Collection and Validation Challenges 60
Conclusion 61
Appendix 62
References 84
State of the Art in Organizational Modeling
Categories of Models: Initial Empirical Results 91
Methodology 92
Results 92
Four-Part Organizing Framework for Models 94
Part II Guide 95
References 96
Verbal Conceptual and Cultural Models 97
Verbal Conceptual Models 97
What Are Verbal Conceptual Models? 97
State of the Art for Verbal Conceptual Models 99
Relevance to Modeling Requirements 100
Major Limitations 102
Verification and Validation Issues 103
Future Research and Development Requirements 103
Cultural Modeling 104
What Is Cultural Modeling? 104
What Is Culture? 105
State of the Art of Culture Models 105
Cultural Inventory Models 105
Dominant Trait Models 109
Semantic Models 113
Cultural Domain Analysis 115
Relevance to Modeling Requirements and Major Limitations 117
Data, Verification, and Validation Issues 118
Future Research and Development Needs 118
References 119
Macro-Level Formal Models 122
System Dynamics Models 122
What Is System Dynamics Modeling? 122
State of the Art in System Dynamics Modeling 129
Early History of System Dynamics 129
More Recent Applications of System Dynamics Modeling 130
Environments for System Dynamics Modeling 133
Relevance, Limitations, and Future Directions 133
Organizational Modeling 135
What Is Organizational Modeling? 135
State of the Art in Organizational Modeling 138
Organization Theory Models 138
Organizational Design Models 141
Relevance, Limitations, and Future Directions 143
References 144
Micro-Level Formal Models 149
Cognitive Architectures 149
What Are Cognitive Architectures? 150
State of the Art 153
ACT-R 155
Soar 155
Epic 156
Cognet 157
Omar 157
Midas 157
Sample 157
Apex 158
Other Architectures 158
Current Trends 159
Verification and Validation Issues 159
Relevance, Limitations, and Future Directions 162
Relevance 162
Major Limitations 164
Future Directions 166
Affective Models and Cognitive-Affective Architectures 167
What Are Cognitive-Affective Architectures? 168
Applications and Benefits of Cognitive-Affective Architectures 171
State of the Art 174
Models of Cognitive Appraisal 175
Models of Emotion Effects on Cognition and Cognitive-Affective Interactions 178
Cognitive-Affective Architectures 180
Relevance to Modeling Requirements 181
Major Limitations 182
Verification and Validation Issues 182
Future Research and Development Requirements 184
Expert Systems 184
What Is an Expert System? 185
State of the Art 188
Expert System Shells and Development Environments 189
Automatic Knowledge Acquisition and Learning 189
Hybrid and Embedded Systems 190
Representing and Reasoning Under Uncertainty 190
Relevance, Limitations, and Future Directions 190
Relevance 190
Major Limitations 191
Future Research and Development Requirements 193
Decision Theory and Game Theory 193
Overview 193
What Are Decision Theory Models? 195
What Are Game Theory Models? 199
Relevance, Limitations, and Future Directions 202
Relevance 202
Major Limitations 205
Future Research and Development Requirements 205
References 206
Meso-Level Formal Models 215
Voting and Social Decision Models 215
What Are Voting Models? 216
State of the Art in Social Decision Modeling 216
Preference Theory 216
Social Choice Theory 217
Strategic Voting 219
Relevance, Limitations, and Future Directions for Social Decision Models 220
Social Network Models 221
What Are Social Network Models? 222
State of the Art in Social Network Models 223
Nodes and Ties 223
Multimode Networks 224
Cohesion Models 225
Centrality Models 225
Equivalence Models 226
Cohesive Subgroup Models 227
Network Evolution 228
Relevance, Limitations, and Future Directions 229
Link Analysis 231
What Is Link Analysis? 231
State of the Art 232
Relevance, Limitations, and Future Directions 234
Agent-Based Modeling of Social Systems 236
What Is Agent-Based Modeling? 237
State of the Art 238
ABM Structural Properties 240
Number of Agents and Cognitive Sophistication 241
Social Sophistication 242
Agents in Grids 242
ABM and Learning 243
ABM and Social Networks 244
ABM Development Issues 245
Relevance, Limitations, and Future Directions 246
Major Limitations 247
Degree of Realism 247
Model Trade-Offs 248
Modeling of Actions 249
Research and Development Requirements 249
Tool Development 249
Forecasting and Possibility Analysis 251
Data Farming 253
Cross-Disciplinary Initiatives 254
Building Expertise 255
Expected Outcomes 256
References 256
Games 261
What Are Massively Multiplayer Online Games? 261
State of the Art 264
Games as an Interaction Medium 264
Games as a Set of Engaging and Immersive Models 264
Games as an Interactive Laboratory 265
Relevance, Limitations, and Future Directions 266
Games as an Interaction Medium 266
Games as a Set of Engaging and Immersive Models 267
Games as an Interactive Laboratory 268
References 269
Common Challenges in IOS Modeling 271
Integration and Interoperability 271
Model Interoperability: Incompatibilities and Functionality Gaps 272
Interface Incompatibility 272
Ontological Incompatibility 274
Formalism Incompatibility 274
Subdomain Gaps 275
Recommendations for Resolving Gaps in Model Interoperability 278
Dealing with Interface Incompatibility 278
Dealing with I-O Format Incompatibilities 278
Dealing with Logical Incompatibilities 280
Dealing with Model Persistence Format Incompatibilities 280
Dealing with Ontological Incompatibility 280
Dealing with Formalism Incompatibility 282
Subdomain Gaps 284
Frameworks and Toolkits 284
General Issues and Requirements 284
IDE Development Goals and Examples 291
Human and System Modeling and Analysis Toolkit 292
Modeling Terrorist Network Evolution 295
Modeling Iraqi Recruiting Activity 297
Advanced Analysis Capabilities 298
Verification, Validation, and Accreditation 301
General Issues: Validation for Use 301
Validation for Understanding and Exploration 304
Validation for Action 305
Military Approaches to Verification, Validation, and Accreditation 313
Validation Issues Specific to Individual Modeling Approaches 317
Validation of Conceptual Models 317
Validation of Cultural Models 318
Validation of Cognitive Models 318
Validation of Cognitive-Affective Architectures 319
Validation of Agent-Based Models 319
Recommendations for Developing and Validating IOS Models 320
Check with Multiple Experts 320
Keep the Model as Simple as Possible for Its Purpose 321
Examine "What Might Be" as Well as "What Is" 321
Use Model Touching for Validation 322
Data Issues and Challenges 324
References 326
State of the Art With Respect to Military Needs 329
Disrupt Terrorist Networks 329
Forecast Adversary Response to Courses of Action 331
Societal Forecasting 332
Crowd Control Training 333
Organizational Design: Force Composition and Command and Control Architecture 334
Reference 336
Addressing Unmet Modeling Needs 337
Pitfalls, Lessons Learned, and Future Needs 339
Pitfalls in Matching the Model to the Real World 340
Model-Problem Mismatch 340
All-Purpose Models That Ultimately Serve No Purpose 341
Verification, Validation, and Accreditation 343
Problems in Designing the Internal Structure of a Model 345
Pitfall of Unvalidated Universal Laws 345
One-Dimensional Models 346
Kitchen Sink Models 347
Pitfalls in Dealing with Uncertainty and Adaptation 348
Unrealistic Expectations 348
Illusions of Permanence 349
Problems in Combining Components and Federating Models 350
Moving from Individual to Collective Action 350
Using Collective Attributes to Predict Individual Action 351
Assemblage of Parts 352
Summary of Future Needs 354
References 355
Recommendations for Military-Sponsored Modeling Research 356
Integrated Cross-Disciplinary Research Programs 357
Independent Research Thrusts 358
Theory Development 358
Uncertainty, Dynamic Adaptability, and Rational Behavior 359
Data Collection Methods 360
Federated Models 361
Validation and Usefulness 362
Tools and Infrastructure for Model Building 362
Multidisciplinary Conferences and Workshops 364
Roadmap for Recommended Research 365
References 369
Appendixes
Acronyms and Abbreviations 373
Exemplary Scenario and Vignettes to Illustrate Potential Model Uses 381
Candidate DIME/PMESII Modeling Paradigms 389
Biographical Sketches of Committee Members and Staff 397
