• A
  • A
  • A
  • ABC
  • ABC
  • ABC
  • А
  • А
  • А
  • А
  • А
Regular version of the site

Measurement of STI

Academic Year
Instruction in English
ECTS credits
Course type:
Compulsory course
1 year, 3, 4 module

Course Syllabus


Decision-making in the science, technology and innovation (STI) domain requires evidence expressed in various indicators for measuring key aspects of generation and dissemination of new scientific and technical knowledge for innovation development and sustainable economic growth. Understanding of such indicators is important to explain how national STI systems are functioning to provide evidence for better policies and informed decision-making. The course introduces the basic approaches and methodologies to design measurement concepts, elaboration of indicators, approaches to data collection and interpretation of the results. Focus is made on the international standards and best national practices of STI measurement. Such include analysis of R&D inputs and outputs, technological and non-technological innovation, scientometrics, analysis of patent data, etc. Moreover, the course is complemented with a review of recent initiatives in indicator development and vision of the future development of STI studies. The course is delivered to the first-year master students of the Master Program ‘Governance of Science, Technology and Innovation’ at the National Research University Higher School of Economics (HSE). The course length is 228 academic hours of which 64 hours are classroom hours for lectures and seminars and 164 hours are devoted to self-study. Academic control forms are two home works, a mid-term test, and a written exam. The course contains six sections, which are mutually exclusive but collectively exhaustive to cover the subject.
Learning Objectives

Learning Objectives

  • Provision students with basic knowledge on international standards, methodology and best practices of STI statistics
  • Provision students with basic knowledge on international standards, methodology and best practices of STI statistics
  • Training analytical and critical thinking skills
Expected Learning Outcomes

Expected Learning Outcomes

  • Knowledge of international standards in STI statistics
  • Knowledge of conventional approaches used for collection and interpretation of statistical data on R&D
  • Ability to select relevant data sources, construct and analyze main R&D indicators
  • Knowledge of conventional approaches used for collection and interpretation of data on publications and patents
  • Ability to construct and interpret bibliometric and patent indicators
  • Knowledge of international conventions and recommendations for collection and interpretation of data on technology development
  • Skills for choosing appropriate data sources and indicators for analysis of Digital Economy, technology intensity and international exchange
  • Knowledge of conventional approaches used for collection and interpretation of statistical data on innovation
  • Ability to select relevant data sources, construct and analyze main innovation indicators
  • Skills for choosing appropriate data sources and indicators for analysis of STI development and use these results for policy advice
Course Contents

Course Contents

  • An overview of STI statistics and indicators
    Statistical studies of STI are based on the complex approach to the measurement of components that constitute national systems of knowledge production and innovation development in accordance with their role in the economy. The lecture introduces the scope and history of STI measurement; it draws a line around its basic principles and international standards in the field and provides links to other areas of statistics. o The basics: national innovation system, linear/non-linear model of innovation, R&D o Scope of STI statistics (e.g. types of STI activities) o History of STI statistics o Basic principles of STI measurement o International standards for STI statistics o STI within the framework of general statistics
  • R&D statistics
    The internationally recognized methodology for collecting and using R&D statistics, the Frascati Manual is a widely accepted essential tool for statisticians worldwide. It includes definitions of basic concepts, data collection guidelines, and classifications for compiling statistics. The lecture introduces basic guidelines for measurement of R&D provided in the FM and implementation of the introduced principles at national and international levels. o Measurement needs and methodological framework o OEСD Frascati Manual o Basic definitions and conventions o Classifications o R&D personnel o R&D expenditure o National and international R&D surveys
  • Measurement of human resources in science and technology
    With the recognition of S&T as an important source of innovation and growth, the need to ensure the adequate supply and career management of R&D and other categories of personnel engaged in scientific performance became one of the key points in the organization of national research systems. The lecture provides an overview of different methodological frameworks that are at hand to help produce relevant sets of data on stock and flows of human resources in science and technology (HRST) as well as indicators on their demographic, career, mobility or skill characteristics. o Definitions and classifications o Measuring stock and flows of HRST o International surveys on careers and mobility of doctorate holders o Measuring skills for innovation
  • Measurement of R&D outputs
    Measuring R&D performance is often associated with assessment of publication and patent activity of research organizations, social groups or individuals. Wider implementation of this approach follows well-established traditions in this field of studies as well as growing demand from policymakers for particular indicators of R&D output. Other characteristics of scientific production (infrastructure capabilities, technology, innovation, social impacts, etc.) are put aside as a self-contained topic. On the other hand, professional discussions on statistical measurement of R&D until recently put aside the problem of accounting for scientific outputs. The lecture is aimed at building a bridge between these traditions and reflecting the scope for measuring scientific performance in general and R&D outputs in particular. o Scope of measuring R&D outputs o Scientometrics and other methods for measuring scientific performance
  • Basics of bibliometric indicators
    Bibliometrics combines a variety of methods allowing to measure research outputs and to analyze social structures of scientific production. Its tools are useful for research evaluation, comparing sciences, understanding tendencies of scientific growth, etc. The lecture gives an introductory overview of the principles of bibliometric analysis, usage of key indicators and recommendations for using data sources. o Indicators and methods o Data sources: an overview o Using bibliometric databases o Data analysis and interpretation
  • Indicators of intellectual property rights
    The lecture is designed to provide students with basic theoretical and practice-oriented knowledge and skills, needed to employ methods of patent analysis for various research tasks. It will discuss research potential of patent statistics, specify the key indicators used in Russian and global studies, provide brief and comprehensive instructions for patent information’ user. Key topics: o Types of IP o Indicators of IP stock and flows o Patent databases o Analysis of patent statistics
  • Mid-term test
  • Statistics on technology
    There is a widespread and increasing interest in the development of indicators for measuring technology development, dissemination and social and economic impacts. Though over the last 50 years relevant statistics allowing for standardized and internationally harmonized measurement of S&T have been established, there is still lack of knowledge on how technologies could be measured. The lecture summarises international state of the art in technology measurement and introduces elements of an integrated approach for technology measurement. 11 o Scope of statistics on technology o Statistics on advanced manufacturing technologies o Biotechnology statistics o Nanotechnology statistics
  • Measuring Digital Economy
    The lecture introduces key principles and best practices in measuring Information Society (IS) and Digital Economy (DE), i.e. ICT supply and demand, infrastructure, usage by businesses, households and individuals, eskills. It starts with an overview of basic definitions of and international statistical standards (OECD, Eurostat, ITU) and continues with approaches to construction and use of indicators to measure different aspects o IS and DE development.
  • Measurement of technology intensity and international trade
    The lecture summarizes key principles for measuring technology intensity and trade in accordance with international standards and best practices in the area and include two subtopics. In many countries high-tech sectors of economy and enterprises are key drivers of economic growth, productivity enhancement, efficient employment, and social development. This section is devoted to review of methods, metrics and available sources of statistical data for high technology analysis, as well as multiple ways of its application. Technology balance of payments (TBP) remains one of the key indicators for measuring international technology trade. This section refers to the key definitions and methodology for calculation and interpretation of TBP data. Key topics: o TBP framework o TBP transactions o Interpreting TBP data
  • Innovation statistics
    Innovation statistics is aimed at measuring new or significantly improved goods or services, technological, organizational, ecological or other advances as well as their role in economic development. The core definition of innovation for the use in statistical measurement was codified 20 years in the Oslo Manual to recognize new technological products and manufacturing processes. Since then its initial coverage was extended from manufacturing to the entire market economy. The lecture introduced basic methodological principles of innovation statistics as well as examples of their implementation at national and international levels. o Methodological framework and measurement needs o OECD Oslo Manual o Basic definitions and conventions o Classifications o Measuring innovation activities o Indicators on objectives, obstacles and outcomes of innovation o National and international innovation surveys
  • New areas for innovation measurement
    Modern innovation studies go far beyond an enterprise perspective covering variety of actors and interactions between them. The aim of the lecture is to give an overview of these studies, providing a background for discussion on further development of innovation measurement. o Open innovation o User innovation o Public sector innovation o Social innovation
  • Use and Interpretation of statistical data
    Variety of statistical indicators on STI development require accurate analysis and explanation for advisory and policymaking at different levels. While manuals allow the ‘routines’ of data collection and interpretation, dynamic economic and social processes as well as specific policy needs require deeper understanding of the facts and tendencies behind the figures. The lecture is focused on available aggregate indicators used for international comparisons and rankings, main sources of information and future perspectives of STI measurement. o Aggregate indicators and rankings o Data visualization and international comparisons o “Blue-sky” issues and the future of STI measurement
Assessment Elements

Assessment Elements

  • non-blocking Participation
  • non-blocking Mid-term test
  • non-blocking Home assignments
  • non-blocking Exam
Interim Assessment

Interim Assessment

  • Interim assessment (3 module)
    0.5 * Exam + 0.2 * Home assignments + 0.2 * Mid-term test + 0.1 * Participation


Recommended Core Bibliography

  • Chesbrough, H. (2012). Open Innovation. Research Technology Management, 55(4), 20. https://doi.org/10.5437/08956308X5504085
  • Emerging technologies: quantitative identification and measurement. (2010). Technology Analysis & Strategic Management, 22(3), 361–376. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsnar&AN=edsnar.oai.ris.utwente.nl.publications.c06bb16c.f7c4.429f.8164.c287be5c61ad
  • European Commission, & Eurostat. (2006). Methodological manual for statistics on the information society. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edseub&AN=edseub.KS.BG.06.004.EN.N
  • European Commission, Eurostat, & Bernard Felix. (2006). Trade in high-tech products. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edseub&AN=edseub.KS.NS.06.014.EN.C
  • Freeman, C., & Fabian, Y. (1987). Output Measurement in Science and Technology : Essays in Honor of Yvan Fabian. Amsterdam: North Holland. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsebk&AN=882088
  • From Too Little to Too Much Innovation? Issues in Measuring Innovation in the Public Sector. (2015). Structural Change and Economic Dynamics, 27, 146–159. https://doi.org/10.1016/j.strueco.2013.06.009
  • Gault, F. (2013). Handbook of Innovation Indicators and Measurement. Cheltenham: Edward Elgar Publishing. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsebk&AN=602821
  • Godin, B. (2017). On the Origins of Bibliometrics ; O początkach bibliometrii. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsbas&AN=edsbas.F608EFCC
  • Gokhberg, L. M., Shmatko, N., & Auriol, L. (2016). The Science and Technology Labor Force : The Value of Doctorate Holders and Development of Professional Careers. [Cham] Switzerland]: Springer. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsebk&AN=1175216
  • Hicks, D., Wouters, P., Waltman, L., de Rijcke, S., & Rafols, I. (2017). Bibliometrics: The Leiden Manifesto for research metrics. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsbas&AN=edsbas.320942A0
  • Leonid Gokhberg, Konstantin Fursov, Ian Miles, & Giulio Perani. (2013). Developing and using indicators of emerging and enabling technologies. Chapters, 349. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsrep&AN=edsrep.h.elg.eechap.14427.15
  • Moed, H. F., Glänzel, W., & Schmoch, U. (2004). Handbook of Quantitative Science and Technology Research : The Use of Publication and Patent Statistics in Studies of S&T Systems. Dordrecht: Springer. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsebk&AN=121551
  • Shalley, C. E., Hitt, M. A., & Zhou, J. (2015). The Oxford Handbook of Creativity, Innovation, and Entrepreneurship. Oxford: Oxford University Press. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsebk&AN=974520
  • TANAKA, N., & SIRILLI, G. (1994). Frascati Manual 1993. Chile, South America. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsbas&AN=edsbas.D22F19DB
  • TANAKA, N., GIOVANNINI, E., WITHERELL, W., & METZGER, J.-M. (2005). Measuring Globalisation. Oecd Handbook on Economic Globalisation Indicators. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsbas&AN=edsbas.879506F6
  • von Hippel, E. A. (2017). Free Innovation by Consumers-How Producers Can Benefit Consumers’ free innovations represent a potentially valuable resource for industrial innovators. https://doi.org/10.1080/08956308.2017.1255055

Recommended Additional Bibliography

  • Angathevar Baskaran. (2016). UNESCO Science Report: Towards 2030 by United Nations Educational Scientific and Cultural Organization (UNESCO), Paris: UNESCO Publishing, 2015, 820pp. Institutions and Economies (Formerly Known as International Journal of Institutions and Economies), (2), 125. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsrep&AN=edsrep.a.umk.journl.v8y2016i2p125.127
  • Auriol, L., Misu, M., & Freeman, R. (2013). Doctorate Holders: Labour Market and Mobility Indicators. Foresight - Russia, 7(4), 16–42. https://doi.org/10.17323/1995-459x.2013.4.16.42
  • Chulhyun Kim, Seungkyum Kim, & Moon-soo Kim. (2011). Identifying Relationships Between Technology-Based Services And Icts: A Patent Analysis Approach. https://doi.org/10.5281/zenodo.1083513
  • Coccia, M. (2004). New models for measuring the R&D performance and identifying the productivity of public research institutes. R&D Management, 34(3), 267–280. https://doi.org/10.1111/j.1467-9310.2004.00338.x
  • European Commission, Eurostat, & Bernard Félix. (2007). High-tech enterprises. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edseub&AN=edseub.KS.SF.07.037.EN.C
  • European Commission, Eurostat, & Tomas Meri. (2008). Trade in high-tech products. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edseub&AN=edseub.KS.SF.08.007.EN.C
  • Fernando Galindo-Rueda. (2013). The OECD measurement agenda for innovation. Chapters, 217. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsrep&AN=edsrep.h.elg.eechap.14427.9
  • Freeman, C. (1994). The economics of technical change. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsbas&AN=edsbas.5AFC99CB
  • Godin, B. (2005). Measurement and Statistics on Science and Technology : 1920 to the Present. New York: Routledge. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsebk&AN=116207
  • Oecd. (2019). Vectors of digital transformation. OECD Digital Economy Papers. https://doi.org/10.1787/5ade2bba-en
  • Open innovation:Research, practices, and policies. (2018). Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsbas&AN=edsbas.884385C4
  • Organisation Coopération, & English Or. English. (n.d.). COMMITTEE FOR SCIENTIFIC AND TECHNOLOGICAL POLICY Working Party of National Experts on Science and Technology Indicators MEASURING R&D IN DEVELOPING COUNTRIES Annex to the Frascati Manual. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsbas&AN=edsbas.9B1DA78F
  • PR Newswire. (2017, April 20). Global Economic Outlook, 2017. PR Newswire US. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=bwh&AN=201704201530PR.NEWS.USPR.BR67394
  • Wipo. (2017). Global Innovation Index 2017. World Intellectual Property Organization - Economics and Statistics Division. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsrep&AN=edsrep.b.wip.report.2017gii
  • Yoshiko Okubo. (1997). Bibliometric Indicators and Analysis of Research Systems: Methods and Examples. OECD Science, Technology and Industry Working Papers. https://doi.org/10.1787/208277770603