University of Melbourne – Hunt Lab for Intelligence Research
« Analytic rigour is central to intelligence work, but there has not been a widely accepted, well
grounded account of what it is and how it might be improved.
We were asked to deliver a report covering (1) the nature of analytic rigour, (2) the factors impacting
it, and (3) opportunities for enhancing it, with our findings to be based on existing academic and
government literature, consultation with experts, and a survey of practicing analysts and managers.
To this end we conducted three main processes:
- A systematic Literature Review;
- An Expert Panel process;
- A Survey of staff in an Australian government agency;
We also conducted a review and synthesis of available government documents related to rigour.
We then further synthesised the results into the findings and recommendations in this report.
Nature. Analytic rigour is best understood, in general, as conducting analytic work in a manner that is
Logical: observing principles of good reasoning and avoiding fallacies;
Objective: being free from influence of values, desires, interests or belief systems;
Thorough: tackling analytic work with completeness and attention to detail;
Stringent: observing relevant rules, guidelines, principles or policies; and
Acute: noticing and addressing relevant issues and subtleties.
We call these the “LOTSA” dimensions. Analytic rigour in intelligence can be further articulated by
describing in more detail how these dimensions apply in various aspects of intelligence work. We use
the LOTSA account to clarify concepts related to rigour such as analytic standards and structured
Factors. We identified many and diverse factors plausibly impacting analytic rigour, falling into six
main categories: analyst attributes, resources, analytic processes, analytic culture, features of the
organisation, and technology. We list and briefly describe these factors.
Opportunities. We define an opportunity to enhance rigour as a potential intervention that is
relatively attractive when considering impact, cost, incidental effects, and timeframe. These are all
difficult to estimate. Nevertheless we identified numerous opportunities in many areas: recruitment,
staff development, resource provision, analytic processes, evaluation and feedback, collaboration,
research, and technology. Just how attractive any one of these is to any given organisation will
depend in part on their particular situation, so we have not further ranked them.
Our research supports some general recommendations:
- Community-wide adoption of a definition of analytic rigour, such as the one provided here.
- Ongoing implementation by organisations of selected opportunities for enhancing rigour.
- Development and adoption of a sound method for evaluating rigour.
- Strengthening of the national capability for research related to intelligence analysis, through
support for a dedicated research entity. »