Magno, 4 August 2011
Mercyhurst College, Erie PA
Advanced Analytic Techniques Course
The purpose of this paper is to assess the value of Risk Analysis as an analytic technique. The method used for the assessment will be a set of evaluation criteria and a Personal Test Case.
The test case will examine risk analysis as a tool to identify terrorist threats to a potential target and to identify the target’s vulnerabilities to an attack. The ultimate goal of the assessment is to give decision-makers the basis to make-cost effective decisions that will ultimately reduce the threat and mitigate its consequences. The four criteria used for the evaluation of the technique are its ability to:
1. Identify credible threats. Has the technique been successful in indentifying groups or entities with the motivation and capacity to do harm to the target?
2. Identify vulnerabilities of the target to attack. Has the technique identified weaknesses in the target that make it vulnerable to attack?
3. Provide information to formulate countermeasures. Has the technique been able to provide the basis to develop countermeasures to the threat?
4. Reduce target vulnerability. Has use of the technique resulted in the reduction of the target’s vulnerability to the threat?
Reduction of vulnerability
Identification of Vulnerability
Formulation of Countermeasures
Identification of Credible Threat
What is the likelihood that the joint US - Pakistani Police Intelligence Center at Rawalpindi will be the target of a terrorist attack within the next 6 to 12 months?
A literature review of eight reliable sources gave a comprehensive critique of risk analysis and its value as an analytic technique to support decision-making. All of the sources were peer reviewed or reliable based on academic or professional standing, subject matter expertise, or proximity and appropriateness. On line sources were evaluated using the Trust Scale and Website Evaluation Worksheet by Dax Norman.
All of the authors with the exception of Douglas Hubbard, in his book “The Failure of Risk Management: Why Its Broken and How to Fix it”, believe that the technique of risk assessment is a valuable tool for decision-makers. The sources consider the technique to be an aid in the efficient management of resources in the avoidance or mitigation of the consequences of threat. The authors presented a wide range of methods to conduct and apply risk assessment.
Douglas Hubbard’s dissent regarding the use and value of the technique was his assertion that users of the technique did not use accurate measurements and mathematically sound computations to assess and manage risk. Further, Hubbard believed that risk assessment was a relatively new concept, without sufficient “best practices” and industry wide proven processes to establish its validity.
Each of the sources presented a slightly different methodology and application of the technique. Jean-Paul Chavas’ Risk Analysis in Theory and Practice examined the use of mathematics and probability theory to increase the accuracy of risk assessments rather than focusing on the use of the technique to develop broad-spectrum risk assessments.
Michael Ronczowski, in his book Terrorism and Organized Hate Crime, explored the use of risk and threat assessments as a result of changes in law enforcement in the aftermath of the 9/11 terror attacks. He believes that terrorism is a law enforcement problem and that risk assessment is a factor in the use of intelligence analysis as a means of forecasting and deterring terrorist activities and events.
Thomas Broder presented a very narrow perspective of risk assessment as it relates to physical security; however, his work Risk Analysis and Security Survey provides the reader with detailed instructions about how to identify the vulnerabilities of a potential terrorist target.
Estimating Terrorism Risk is a work sponsored and published by the Rand Corporation. Its authors are expert analysts and specialists in international relations. The article provides the reader with a technique for an assessment of risk and risk management at the strategic level. The source differs greatly from Broder’s Risk Analysis and the Security Survey, which has a very narrow tactical focus, assessing risk to mitigate threat and enhance security for a specific target.
Robert Clark’s, Intelligence Analysis: A Target Centric Approach, Richards Heuers’ and Randolph Pherson’s, Structured Analytic Techniques for Intelligence Analysis, and Heather Wynen’s, Structured Analytic Techniques for Intelligence Analysis, are all “ How to” guides presenting analytic techniques to industry professionals. Each source presents a slightly different approach to risk analysis and taken in total offer a broad view of risk analysis.
In spite of variations in methodology and focus, the sources present systematic approaches to risk analysis. All of those approaches include the identification of a threat or threats, analysis of the vulnerabilities of the object of the threat, and consideration of its consequences or impact.
Risk Analysis is a technique used by the military, law enforcement, and the private sector to reduce uncertainty in decision-making.
Risk analysis gives the user the opportunity to identify specific threats to an entity allowing decision makers an opportunity to deter, avoid, or mitigate the impact of the threats (Broder, J.F., Risk Analysis and the Security Survey).
The technique identifies strengths and weaknesses in an entity giving the user an opportunity to devise efficient, effective methods to strengthen weaknesses, adopt countermeasures to the threats, or adopt courses of action to avoid or overcome threats (Chavas, J.P. Risk Analysis in Theory and Practice, Broder, J.F., Risk Analysis and the Security Survey).
The technique uses wide varieties of methods to calculate data and quantify risk allowing the decision-maker to develop effective cost efficient options or counter measures (Hubbard, Douglas, W., The Failure of Risk Management: Whay it doesn’t Work and How to Fix It, Chavas, J.P. Risk Analysis in Theory and Practice, Wynen, H., Risk management and intelligence in review Canada, Customs).
Risk Analysis as an analytic technique has weaknesses that the user must consider.
Risk analysis is a snapshot in time. The evaluation and analysis process must become a constant ongoing process (Broder, J.F., Risk Analysis and the Security Survey).
Risk analysis is dependent on the skill, knowledge, experience, and motivation of the user. The most methodical statistic based calculations using mathematically sound methods still rely on the informed assessment of the subject matter expert (Chavas, J.P. Risk Analysis in Theory and Practice).
The human mind is limited in its capability and capacity to process information. The process requires specialization and collaboration to be effective (Chavas, J.P. Risk Analysis in Theory and Practice, Heuer, Richards, J., The Psychology of Intelligence Analysis).
The process will generate a product that will reduce decision-makers uncertainty about a given requirement or analytic question. The technique considers the basic elements of risk: threat, vulnerability, and consequences, and provides the user with the basis for making an effective and efficient decision to reduce or avoid threat.
1. Formulate a specific analytic question
Analyst(s) work with the decision-maker(s) to develop a clear well defined analytic question so that both the analysts and the decision-maker understand and agree to the objective of the analysis. A specific well-defined analytic question ensures that the analysis meets the needs of the decision-maker(s).
2. Identify the scope of the question and the elements of the problem posed by the question Deconstruct the question or requirement. Identify the critical issue or problem, and any issues or sub-problems that are associated with it and have an affect or impact on it. Identify information needs or gaps. What is the object or target of the risk assessment? What are the specific dangers or threats to the target? What are the vulnerabilities of the target? What are the consequences?
3. Formulate a data collection plan
Plan the collection of data. Identify likely sources for data that will provide the information needed to answer the analytic question or fill the information gaps, and identify the method of collection.
4. Collect and evaluate the data
Collect the data, evaluate it for relevance and reliability.
5. Organize and analyze the data
Organize the relevant reliable data. Categorize the data and organize it so that it is readily retrievable. The analyst reviews the data to identify elements and patterns applicable to the risk assessment.
6. Identify threats and vulnerabilities
Review the data and conduct threat and vulnerability assessments to identify specific threats or vulnerabilities and their consequences.
7. Develop options to mitigate or neutralize risks
Formulate options and measures to avoid, mitigate, or neutralize the risk.
8. Develop recommendations for cost effective methods to mitigate risks
Evaluate the target vulnerabilities, prioritize the threats to the target and their consequences, and develop recommendations to reduce the target's vulnerabilities against the threats making the best use of available resources.
9. Use expert opinion to review validity of analysis: process and recommendations
Conduct a review of the process and product by experienced analysts and subject matter experts to ensure that the process and product is valid and reliable. Conduct additional analyses, revise, and re-work the problem as is necessary.
10. Deliver analytic product to decision maker
The following is diagram is a concept map of the risk analysis process.
1. Broder, J. F. (2006). Risk analysis and the security survey (3rd ed.). Burlington, MA: Butterworth-Heinemann
The analytic question deals with the physical security of a government facility. Broder’s book is very relevant. He provides step-by step procedures to assess facility security and detailed checklists to identify vulnerability.
2. Clark, R. M. (2007). Intelligence analysis: A target-centric approach (2nd ed.). Washington, D.C.: CQ Press
Richard Clark’s book was informative and provided a good insight into the concepts of risk analysis and the relationship of the motivations and capacity of the threat to the vulnerability of the target.
3. Heuer, R. J., Pherson, R. H. (2011). Structured analytic techniques for intelligence analysis.Washington, D.C.: CQ Press.
This source, written by two well know subject matter experts, is an excellent and important guide for intelligence analysts. The source offers a simple and clear explanation of a wide range of important analytic techniques. The chapters, Selecting Structured Techniques,
Decomposition and Visualization, and Evaluation of Techniques were especially helpful in the development of this paper and the ranking and scoring of criteria used in the assessment.
4. Willis, H. H., Morral, A.R., Kelly, T.K. and Medby, J.J. (2011) Estimating Terrorism Risk, Santa Monica, Calif: RAND Corporation.
Estimating Terrorism Risk has minimal application to the personal test case but it gives an excellent explanation of the use of risk analysis in assessing and managing risk at the strategic level. It is worthwhile and informative reading.
This paper examines a personal test case to evaluate the effectiveness of risk analysis as an analytic technique. The personal test case was formulated to answer an analytic question using risk analysis.
The exercise was an excellent learning experience. It gave the writer the opportunity for hands on practice conducting risk, threat and vulnerability assessments. It was also an opportunity to select and develop measurement methods to support the assessments.
The following steps were used to conduct the risk assessment and to answer the analytic question: What is the likelihood that the joint US - Pakistani Police Intelligence Center at Rawalpindi will be the target of a terrorist attack within the next 6 to 12 months?
Formulate a specific analytic question.
The analyst or the head of the analytic unit working with the decision-maker identified a specific issue of concern: What is the likelihood that the joint United States (US) - Pakistani Police Intelligence Center at Rawalpindi (UPIC) will be the target of a terrorist attack within the next 6 to 12 months? The product of risk analysis is time sensitive. The pilot project has the potential to be an effective tool in combating terrorism in the region and therefore it is likely to be seen as a symbolic target and draw the attention of terrorist groups. James Brorder ( Risk Analysis and the Security Survey) makes the point that threats, vulnerabilities, and consequences change over time. Risk analyses should be conducted periodically. Therefore, the focus of the analysis a period of 6 to 12 months.
Identify the scope of the question and the elements of the problem posed by the question.
The analytic question deals with the security of the UPIC facility. Sub-problems are the physical security of the UPIC facility, its personnel its operations, and information. The consequences of a successful attack may include the loss of life and resources, political, and strategic impact.
Formulate a data collection plan.
The process required that data be collected to identify credible threats and vulnerabilities of the subject target. Collection was restricted to open source data. Data was collected from reliable sources about the most prominent and capable terrorist groups with the ability and desire to conduct a successful attack at the site. Information about the latest trends in terrorist tactics, as well as historic information to identify patterns of behavior of groups identified as credible threats was collected from reliable internet sources. Information about target security and vulnerability was collected using security site surveys obtained from subject matter experts in physical security. In this case, security check lists furnished in Risk Analysis and the Security Survey by James Broder were used to identify target vulnerabilities.
Collect and evaluate the data
Information was collected from internet open sources. The information was evaluated for its relevance and application to the analytic question posed. The relevant data was then evaluated for reliability using the Dax Norman Trust Scale and Web Site Evaluation Worksheet (Appendix A).
Organize and analyze the data
Reliable relevant datawas categorized and organized by type, date, source, and reliability, and entered into a Microsoft Excel spreadsheet for easy retrieval and manipulation.
Identify threats and vulnerabilities
In the personal test case, the threat is the possibility of a terrorist attack on the US-Pakistani Joint Intelligence Center at Rawalpindi (UPIC). The intelligence center and the program are fictitious entities. The model for the physical layout of the center and its security profile (fig.1) is the Starfighter base from the 1984 film, The Last Starfighter, produced by Universal Pictures (/title/tt0087597/).
* Note: Sources for footnotes 1-6 are found in Appendix BFig.2).
Criteria Evaluation Scale for Capability
Frequency of Operations (F)
Coordination ( C )
Operational Capability (OC)
Personnel Expertise (E)
Target Selection (T)
Capability= ( F + C + OC + E + T) / 5
Consistent successful operations over 48 months or more
Coordinated execution of Multiple personnel ( 8 or more) or units (2 or more) or more in simultaneous operations
Sophisticated and high quality weapons and large support base in a variety of geographic areas and successful complex attack scenarios
Dedicated personnel with specialized expertise and training
Successful attacks against hard and soft targets
Consistent successful operations over 24 to 48 months
Coordination of multiple personnel (6 or more) or units (2 or more) in a single operation
High quality weapons strong base of support in limited area successful attack scenarios
Well trained personnel with limited expertise in specialties - Use of Ad Hoc personnel for specialized operations
Limited success against hard targets high number of successes (12 or more) against soft targets
Sporadic successful operations over 24 to 48 months
Coordination of small unit 2-4 in a single operation
Poor quality and improvised weapons weak or limited support base in a wide geographic area
Dedicated personnel with minimal specialty expertise
Successful against soft targets only
More unsuccessful than successful operations over 24 to 48 months
Small unit 1-2 personnel in single operation
Poorly armed with low quality weaponry, weak support base in a limited area
Limited personnel with general experience and knowledge
Sporadic success against soft targets
Frequency of Operations
Lashkar I Jhangvi
Motivation is defined as a reason and desire to act. The criteria used for measuring the reason for a terrorist groups’ selection of the object of their attack and their desire to act are described in the matrices labeled Figures 5 and 6.