Business Case, Feasibility, and Research Community Impact
Prepared by: Strategic Planning Consultants Ted Kraver, President Mark Goldstein, Team Leader Oris Friesen, Senior Researcher Dee Andrews, E-Learning Consultant
August 15, 2005 (V03A Draft)
Institute for Defense Analyses (IDA) E-Learning Research Portal Study Business Case, Feasibility, and Research Community Impact
Prepared for: The Institute for Defense Analyses (IDA) - http://www.ida.org/ Dexter Fletcher, IDA Science and Technology Division Phone: 703-578-2837, E-mail: firstname.lastname@example.org Robert Foster, Office of the Secretary of Defense (OSD) Phone: 703-588-7420, E-mail: email@example.com
This report was researched and prepared by Strategic Planning Consultants for the Institute for Defense Analyses (IDA). However any findings, opinions, conclusions or recommendations expressed herein are those of Strategic Planning Consultants and do not necessarily reflect the views of the Institute for Defense Analyses (IDA). All trademarks and copyrights are the property of their respective owners. Unauthorized redistribution of this report is a violation of U.S. and international copyright laws.
Note: This report is in a developmental draft stage and intended for limited distribution for team review and comment. Some quotes, document excerpts, and figures currently included may need author and/or publisher clearances to be utilized in the final published version or may need to be excised from this report if such permissions are not sought or granted.
Please do not distribute externally at this time!
Institute for Defense Analyses (IDA) E-Learning Research Portal Study Business Case, Feasibility, and Research Community Impact Table of Contents Page E-Learning Research Portal (ELRP) Project Overview 5 ELRP Executive Summary 6 E-Learning Research Portal Impact and Business Case 11 E-Learning Research Portal Project Overview 21 E-Learning Research Portal Project Phases 22 Phase 0: Business Case and Feasibility Study Phase 1: E-Learning Research Portal Pilot Phase 2: E-Learning Research Portal Implementation Phase 3: Database and Portal Maintenance and Support E-Learning Research Portal Project Roadmap 23 E-Learning Research Portal Project Projected Costs 24 E-Learning Researcher Knowledge Clusters and Acquisition 26 Individual and Institutional Knowledge Communities of Practice (CoP)
Knowledge Acquisition, Analysis, and Tagging
E-Learning Research Ontology and Taxonomy 33 E-Learning Research Metadata and XML Coding 45 E-Learning Research Portal Organization and Functionality 48 E-Learning Research Portal Special Purpose Software Tools 53 Primary Software for Possible ELRP Utilization 56 Secondary Software for Possible ELRP Utilization 66 Additional Software Relative to Specific Functionality 70 Portal, Communities of Practice (CoP) and Collaboration Tools 100
Institute for Defense Analyses (IDA) E-Learning Research Portal Study Business Case, Feasibility, and Research Community Impact Page Appendix A - E-Learning Research Centers & Communities of Practice (CoP) 108 U.S. Government Military Research Programs 109 U.S. Government Dept. of Education Research Programs 113 Foreign Government Research Programs 114 Academia-Based Research Programs 120 Industry Research Programs 128 Non-Profit Research Programs 130 E-Learning Research Communities of Practice (CoP) 132 Appendix B - E-Learning Research Resources 133 E-Learning Research 134 Ontologies and Taxonomies 141
Metadata and XML Coding 146
Text Mining and Semantic Analysis 148 Knowledge Visualization, and Navigation 150 Communities of Practice (CoP) and Collaboration 153 Appendix C - Companies and Products Referenced 155 Appendix D - SPC Project Team Background 160 Charts and Figures Comparison of Potential ELRP Development and Sustaining Models 20 E-Learning Research Portal Project Roadmap 23 E-Learning Research Portal Projected Project Costs 24 ELRP Operational Budget Scenario 2005-2009 25 E-Learning Industry Enterprise Value Chain 25 Defining E-Learning Research Communities of Practice Knowledge (CoP) 28 Building Communities of Practice 29 Communities of Practice Knowledge Architecture 30 Information Engineering (InfoE) Layers Model 31 E-Learning Research Knowledge Acquisition, Analysis and Tagging 32 Taxonomy of Intelligence Analysis Variables Example 44 E-Learning Research Portal Organization 48 Collaborative Technologies for Enterprise Knowledge Work 50 Sample Software Function Ranking Chart 55 Various Software Ranking Charts, Architecture Diagrams, and Application Screenshots
IDA E-Learning Research Portal (ELRP) Project Overview
E-Learning Research Portal Organization
Knowledge Acquisition, Analysis and Tagging
Operational Capability: The Institute for Defense Analyses (IDA) ELRP would provide the military and other e-learning research communities with a rich and highly relevant e-learning research database, coupled with sophisticated retrieval and visualization tools. E-learning research investigators and decision makers would have access to a meta tagged database that can be autonomically or manually updated on a continuous basis covering e-learning research institutions and practitioners, related Communities of Practice (CoP), topical news, and selected relevant literature. It will help to locate, characterize, and connect principal investigators and lead players within various e-learning research CoP. Search will support a variety of querying, navigation, and presentation functions and capabilities, including clustering of relevant results, advanced 3-D visualization, and user designated pathways to relevant knowledge. It also provides for extractions of subsets of the database for other analysis.
Based on a custom e-learning research ontology, taxonomy, and eXtensible Markup Language (XML) variant for metadata tagging, a survey team will identify and engage a large number of e-learning research organizations and individuals to initially populate the database, augmented by autonomic semantic analysis of survey results, mining of CoP, as well as indexing of selective web sites, news feeds, and blogs relevant to e-learning research domains. Users will be able to identify prior and current relevant research activities, possible organizational partners, key individuals for research background and collaboration, and potential funding sources. The aggregate database and associated metadata will form a unique and uniquely valuable resource for the e-learning research community.
Proposed Technical Approach: The project team will develop a specific ontology, taxonomy, and XML variant for semantic analysis and meta data tagging of e-learning research sources and resources. A web portal will be developed as a primary functional framework, including support for the coordination, communication, and collaboration of e-learning research CoP. The project team will evaluate and select a single suite or several complementary and compatible applications from currently available commercial software tools applicable to the project’s distinctive database, analytic processing, querying, results clustering, navigation, and visualization needs. The abilities of these commercial tools already being employed within larger government and industry enterprises to perform the necessary key functions such as taxonomy authoring, XML editing and management, unstructured data analysis and meta tagging, meta data repository management, sophisticated querying, as well as clustering of results, navigable 3-D visualization, and other presentation modes are sufficient to meet the project’s needs, minimizing the overall costs, risks, and effort for the ELRP’s pilot phase and full implementation to follow. DRM and security protocols will provide appropriate access to information.
Deliverables: Phase 0: ELRP Business Case, Feasibility, and Research Community Impact Study and Consultations
Phase 1: E-Learning Research Portal (ELRP) Pilot
Phase 2: E-Learning Research Portal (ELRP) Implementation, CoP Engagement, and Launch
Phase 3: ELRP Database and Portal Maintenance and Support with Transfer to Sustaining Organization
Costs and Schedule:
Phase 0: To June 2005 at < $50K Total
Phase 1: July 2005 - June 2006 at $165K Total
Phase 2: July 2006 - June 2007 at $520K Total
Phase 3: > July 2007 at $150K per Annum
Contact Information: Mark Goldstein, ELRP Project Team Leader
ELRP Executive Summary A critical step to achieving greater information agility and analytic discovery is effective management and exploitation of information content. Content Management addresses the issues associated with organizing, locating, and accessing information. To exploit the value of emerging data mining and knowledge discovery tools, we must put agency information into both a form and an environment where it can be more effectively managed. The commercial sector is addressing information agility, data interoperability, and content management through a combination of technology and standards, including indexing, federated databases and taxonomies, and data tagging. DIA will use commercial solutions, such as eXtensible Markup Language (XML), to manage its information to improve our ability to locate data and create knowledge through analytic tools and processes.
U.S. Defense Intelligence Agency (DIA) Strategic Plan 2004-2009
(http://www.dia.mil/internet_strat_plan/52712_cov.html or http://www.dia.mil/internet_strat_plan/strat_plan.pdf)
The U.S. military has long recognized the growing dependence of its decision-makers and warfighters on information generated and shared across worldwide networks, and has invested substantially in advancing the capabilities and security of information technology for use in command, control, communications, and computer (C4) systems. With their enormous training needs, they have also invested in and remained at the forefront of e-learning research, developing a variety of initiatives and standards while driving deployment throughout its ranks across the organization.
Today, the Department of Defense (DoD - http://www.defenselink.mil/) is using computer technologies and advanced instructional methods and design so as to improve the efficiency and effectiveness of defense training. The Advanced Distributed Learning (ADL - http://www.adlnet.org/) initiative enables instructional materials and performance support aids to be accessed by DoD personnel at times and places of their choosing. Efficiency and effectiveness is driven by investments in technology-based instruction through the development of reusable, web-available "instructional objects," consisting of a self-contained course or set of instructions for military job-related activities.
The Institute for Defense Analyses (http://www.ida.org/) is a federally funded research, development, and study center established to assist the Office of the Secretary of Defense, the Joint Staff, the Unified Commands, and Defense Agencies in addressing important national security issues, particularly those requiring scientific and technical expertise. IDA also conducts related research for other government agencies on national problems for which the Institute’s skills and experience are especially suited.
IDA has established an ADL Co-Laboratory, which is developing guidelines for creating effective, reusable instruction objects. The laboratory organizes and hosts "plugfests" in which government and commercial providers demonstrate and assess the instructional objects they are developing under the ADL program. The Co-Laboratory further acts as a clearinghouse for information, data, and lessons learned. IDA is also helping develop a web-based tutoring system that uses ADL technologies. This system, the "human use regulatory affairs advisor," will provide tutorials and decision aids concerning procedures and protections required when using human subjects in DoD research.
IDA has been investigating the feasibility and considering funding the development of an E-Learning Research Portal (ELRP). The ELRP would provide the military e-learning research community a rich and highly relevant e-learning research database, coupled with sophisticated retrieval and visualization tools. Military R&D investigators and decision makers, as well as potentially broader e-learning research and end user communities, would have access to a meta tagged database covering the universe of e-learning research institutions and organizations, individual practitioners, related Communities of Practice (CoP), topical news, and selected relevant literature. The database would be autonomically as well as manually updated on a continuous basis. It would include extensive profiles of e-learning research performing organizations and practitioners (in government, industry, and education) detailing their focus areas, core competencies, and available resources (human, bibliography, laboratory, linkages, etc.). The database would also serve to help locate and characterize principal investigators or lead players within various CoP across the e-learning universe. The ELRP’s web-based user interface would allow for a variety of querying capabilities and modes with the resultant data sets presented in a variety of forms including advanced navigable 3-D visualizations with clustering of relevant results and user defined and controlled pathways to help lead them to relevant knowledge. Alternatively, extractions and subsets of the database could be readily pulled for analysis and post-processing by other tools and means.
To further explore these ends, IDA has engaged Strategic Planning Consultants in conjunction with International Research Center, both of Phoenix, Arizona, to carry out this study on IDA’s behalf. This initial ELRP Phase 0 has undertaken research and analysis, while engaging critical elements of the e-learning research community, leading to and concluding with release of this E-Learning Research Portal Study: Business Case, Feasibility, and Research Community Impact report. This report details the potential of the proposed ELRP, including its business case, feasibility, target deliverables, research community impact, and project implementation issues and options. A summary PowerPoint presentation complements this report and can be used to pursue further support and initial funding to develop and launch the proposed ELRP, while engaging the target e-learning research community.
Based on these findings, IDA will decide whether to proceed with the proposed Phase 1 ELRP Pilot and potentially a full ELRP implementation to follow. Other practical scenarios are presented and compared alternative organizations or funding sources that could support the further development, implementation, and sustaining of the ELRP. The projected Phase 1 ELRP Pilot would proceed as soon as practical to conduct associated research, engage the target e-learning researcher community, continue taxonomy and ontology development, and implement a fully-functional prototype portal populated with live, current data for development and demonstration of the database and query/visualization capabilities. User feedback would be employed to refine the e-learning research taxonomy and XML variant, expand the data sources and live feeds, as well as optimize data searching and presentation capabilities. A related Communities of Practice (CoP) would be activated and cultivated around the project pilot portal to gain user feedback and coalesce researcher interest and participation. After the pilot portal had sufficient operational experience, at IDA or another primary pilot sponsor’s discretion, the pilot would then be followed by a Phase 2 full system implementation. Phase 2 tasks would include extensive research and researcher surveying, incorporation of additional data sources, development and mining of relevant CoP, database completion and update protocols, enhancement of query/visualization capabilities, and complete deployment and launch. Finally, Phase 3 activity would include ongoing sustaining operations, updates, and evolution.
A contractor’s project team would initially develop a specific ontology and taxonomy of e-learning research driving the adaptation or development of an XML variant for metadata capture relative to e-learning research sources and resources. A survey team would identify and engage a large number of e-learning R&D organizations and individuals, characterizing and coding them for database entry as well as involving them in a needs analysis and to validate portal design concepts. The meta data repository would be kept current by online end user updating capabilities, additional periodic surveying, incorporation of relevant news feeds, as well as targeted, periodically scheduled web crawling and indexing of selected domains and subdomains. The results would be post-processed by automatic text analysis and the meta tagging of unstructured textual content from these sources, as well as from relevant mailing lists, discussion groups, and existing Communities of Practice (CoP). The aggregate database results and associated metadata would form a unique and uniquely valuable resource for the e-learning community. By including both intra- and inter- community linkage players, the ELRP would help to identify prior and current relevant research activities, possible organizational partners, key individuals for research background and collaboration, as well as potential funding sources. This unique system and applied toolset could eventually be expanded for use in other advanced research and development fields, as well as adapted to other domains and problem sets.
Fortunately, there are quite a variety of currently available commercial software tools that can deliver an unprecedented level of portal functionality and automation, as well as quite a number with possible applicability to the project’s analytic and visualization needs from which the project team would carefully compare and select a primary suite and complementary capabilities to perform the key functions such as ontology and taxonomy authoring, unstructured data analytics and meta tagging, as well as clustered results visualization and integration to the ELRP’s underlying portal platform. These special tools and capabilities are already being widely employed within larger enterprises as part of an overall or specialized knowledge management strategy and supported by an active and evolving industry sector delivering a breadth of rich and robust applications across multiple tasks and domains.
Strategic Planning Consultants has initially identified five software suites that can perform most or all of the required special functions, making them prime candidates for anchoring the ELRP’s special capabilities characterized as Ontology and Taxonomy Authoring and Maintenance, Text Mining/Semantic Analysis, Meta Tagging and Database Repository, and Knowledge Visualization and Navigation. A selected product from this or an expanded group will be potentially complemented by an additional software product if gaps exist. Beyond that, the ELRP will largely rely on a mix of common Open Source and off-the-shelf commercial software for developing the portal and supporting its primary framework and functionality, including support for the coordination, communication, and collaboration of e-learning research Communities of Practice (CoP).
Software function ranking charts were developed to show the relative strength of leading software candidates across the six primary functional categories. An example diagram follows, illustrating how combining Semagix’s Freedom Suite with Lumen’s Lumenation Portal Framework enables a full range of special functions and solutions to “snap” into the basic portal structure and capabilities, readily integrating together. Clearly these two products in combination have little functional overlap while together delivering strength in all six of the primary functional categories. During the pilot implementation, the project team would further research, formally evaluate, recommend, and adopt Open Source and commercial software tools to best meet the project’s IT framework, defined CoP needs, and overall budget. Confidence is high that appropriate suites, toolsets, and add-ons for the various project sub-tasks can be selected and licensed for optimal integration, speedy implementation, and ongoing support, minimizing the efforts and risk for project implementation.
This E-Learning Research Portal (ELRP) would have a variety of tangible and intangible benefits including a substantial impact on the e-learning research community. The ultimate beneficiary would be the e-learning industry as a whole yielding considerable improvements to U.S. military e-learning capabilities, effectiveness, and efficiencies. Some of the expected tangible benefits include:
Improvement in access, efficiency, and effectiveness supports queries from practicing e-learning researchers;
Depth and quality of search assures good grounding for initiatives and reduced redundancy;
Significant opportunity benefits and value arises from connecting collaborative players;
The cast for increased federal funding for e-Learning research will be supported by assessments and studies facilitated by ELRP;
ELRP will expand Communities of Practice (CoP) utilization among e-learning research stakeholders and increase the utility and utilization of their collaborative communication;
Improved knowledge accessibility greatly increases the value of the collected content;
Underlying ontology and taxonomy efforts will define, refine, and aid e-learning research;
Improved e-learning research will yield improved e-learning products and deployment which will in turn better serve military and government simulation and training needs;
Improved e-learning research deployment will drive economic engines in the e-learning industry;
A sustainable ELRP is relatively low risk by using existing and emerging toolsets and designing an organizational structure from multiple practical scenarios for long-term maintenance, evolution, and support;
ELRP will demonstrate proof of concept for applying these techniques across other scientific research domains with its use of ontologies and taxonomies integrated with text mining, semantic analysis, and knowledge extraction capabilities; presenting query results in a knowledge visualization and navigation framework; and developing understanding of applicability, benefits, cost basis, and many other issues.
The ELRP development project’s full life cycle costs and attendant risks must be assessed to properly gage the business case. The critical success issues are the project’s value proposition, implementation feasibility, and costs, as well as practical scenarios for sustainability into perpetuity. This ELRP Phase 0 study uncovers significant existing problems in awareness and communication of e-learning research activities among disparate organizations and groups, explores the opportunities and benefits presented by e-learning research visibility and enhanced CoP, and determines how the proposed portal can offer general aid to the e-learning research community and solutions to these problems, advancing the state of the art in the process.
ELRP supported Communities of Practice (CoP) will link together to create the necessary system to solve many of today’s e-learning research problems, such as awareness of and collaboration between various research efforts and initiatives. Creation of the ELRP will greatly enhance researchers’ opportunities for success in unique discovery, collaboration, results verification, and eventual deployment. The resultant advances in the relevant e-learning research base of knowledge and the facilitated connections between and within relevant CoP will serve to help advance e-learning’s state of the art. The result is significant benefits to researchers, other e-learning stakeholders, an emergent e-learning industry, and the knowledge worker in general.
Strategic Planning Consultants has found through the Phase 0 study activities and its findings as detailed in this report that the development of the proposed ELRP is both practical and feasible. The ELRP will be of significant value and positive impact to military and government e-learning deployment and training effectiveness. It will clearly serve both the government’s direct training and workforce issues, as well as support U.S. leadership in e-learning R&D, serve domestic industry development needs, and help insure the success of U.S. products in domestic and international markets. Therefore it is strongly recommended that IDA consider undertaking at the earliest practical time the funding, development, deployment, and testing of the Phase 1 E-Learning Research Portal Pilot, to be followed at its successful conclusion with the full Phase 2 E-Learning Research Portal Implementation and follow on Phase 3 Database and Portal Maintenance and Support.
As Vannevar Bush, who was Director of the U.S. Office of Scientific Research and Development for President Franklin D. Roosevelt, and thus coordinated the activities of some six thousand leading American scientists in the application of science to warfare during World War II, so aptly stated:
Professionally our methods of transmitting and reviewing the results of research are generations old and by now are totally inadequate for their purpose. If the aggregate time spent in writing scholarly works and in reading them could be evaluated, the ratio between these amounts of time might well be startling. Those who conscientiously attempt to keep abreast of current thought, even in restricted fields, by close and continuous reading might well shy away from an examination calculated to show how much of the previous month's efforts could be produced on call. Mendel's concept of the laws of genetics was lost to the world for a generation because his publication did not reach the few who were capable of grasping and extending it; and this sort of catastrophe is undoubtedly being repeated all about us, as truly significant attainments become lost in the mass of the inconsequential.
The difficulty seems to be, not so much that we publish unduly in view of the extent and variety of present-day interests, but rather that publication has been extended far beyond our present ability to make real use of the record. The summation of human experience is being expanded at a prodigious rate, and the means we use for threading through the consequent maze to the momentarily important item is the same as was used in the days of square-rigged ships. But there are signs of a change as new and powerful instrumentalities come into use.
From As We May Think by Vannevar Bush, The Atlantic Monthly, July 1945