Currently, there is an increase in investments and access to Information Technologies, with an increase in their applicability in evaluative studies in health to improve and speed up data, subsidies to decisions and equity. The objective of the study was to evaluate the incorporation of Information Technology in the evaluation of Primary Health Care in Brazil. Considering the relationships between technologies and subjects (sociomateriality), the general objective of this thesis was to evaluate the incorporation of Information Technology in the External Evaluation of the Program for Improving Access and Quality in Primary Care. To meet this objective, two types of studies were carried out, the Methodological Study and the Case Study. The Methodological Study proposed to build and validate the content of an item matrix of an instrument for the evaluation of External Evaluation (meta-evaluation). Seven experts with evaluation expertise and 16 evaluating researchers participated in the study. To assess the relevance of the items, face-to-face workshops were held to reach consensus. In assessing relevance and representativeness, the Content Validity Index and Interrater Agreement were calculated. As for the reliability assessment, an analysis of the internal consistency determined by Cronbach's alpha was performed. For the assessment of clarity, understanding, a Focus Group was conducted with the application of an operational instrument based on the matrix. The validity matrix consisting of 38 items, distributed in 24 indicators of the 4 quality standards of the Joint Committee on Standards for Educational Evaluation (utility, accuracy and feasibility) distributed in the 06 evaluation steps of the Centers for Disease Control and Prevention. The final instrument to be used in the meta-assessment contains 3 sections and 16 items.

The Case Study had a qualitative approach with triangulation of documentary interview and Focus Group analysis from July 2018 to July 2019. The sample size was 54 subjects (7 technicians from the Ministry and 47 evaluating researchers). The data were categorized by quality standards using the Atlas ti 7.1 software followed by Bardin's content analysis. As a result, it was shown that Information Technology made the assessment feasible; minimized errors; made available a database for team certification, health policies and research; contributed with accuracy in the homogenization of the collection, validation; field management and proved to be appropriate to a more coordinated and collaborative scenario, making the assessment more consistent with the potential for greater use of results. In addition, factors such as difficulty in handling, inoperability of the system and lack of investments in equipment and human resources may represent possible restrictions for the efficiency of the application of technology in the evaluation. The thesis under development, aligns instrument validation with the Case Study, signaling that Information Technology has the potential to reach quality standards, creates synergism between them and is an innovation in the Primary Health Care assessment scenario. The thesis is part of the Project “Evaluation of the External Evaluation of the Program for Improvement of Access and Quality of Primary Care (PMAQ — AB) in Brazil: a Case Study” partially funded by the Ministry of Health (MS / UFRN / TED nº 106 / 2015) and the National Council for Scientific and Technological Development ((notice 09/2018) and approved by CEP-HUOL (CAAE 84537418.1.0000.5902).