SCRAPERS

1

Scrapers

Team Members:

Guankun Li : guankun@uchicago.edu

Jiazheng Li : jiazheng123@uchicago.edu

Tianyue Cong : tianyuec@uchicago.edu

Weiwu Yan : weiwuyan@uchicago.edu

​

GitHub: https://github.com/macs30122-winter24/final-project-scrapers.git

MACS 30122

The Impact of National Science Foundation (NSF)�Funding on Academic Output of Scholars

SCRAPERS

2

MACS 30122

Background

Funding facilitates the acquisition of resources necessary for research. This project examines the pivotal role of the National Science Foundation (NSF) funding in shaping the academic output of scholars within the realms of Behavioral and Cognitive Sciences.

​

The focus on Behavioral and Cognitive Sciences is deliberate and strategic. This domain stands at the crossroads of natural sciences and social sciences. The inherent diversity within this field, with its clear subdivision into various subfields, provides a unique opportunity to analyze the impact of funding across different areas of study.

SCRAPERS

3

MACS 30122

Social Science Significance

In the contemporary landscape of scientific development, research funding has emerged as one of the most crucial public resources. Research funding, by allocating financial resources to research projects, institutions, and researchers, is instrumental in catalyzing innovation, facilitating growth within academic disciplines, advancing the careers of scientists, and contributing to the broader socio-economic development (Lane, 2009). The contribution of research funding to knowledge production is substantial.

For instance, between 2009 and 2010, out of 2,060,838 research papers indexed in the Science Citation Index (SCI), 1,165,276 papers (56.54\%) received at least one form of research funding support (Mahesh, 2012). This underscores the significance of evaluating the outcomes and impact of research funding.

SCRAPERS

4

MACS 30122

Research Questions

RQ1

Does NSF funding increase the quantity or quality of academic output of scholars?

RQ2

Which subfields within Behavioral and Cognitive Sciences are more likely to be awarded funding?

RQ3

Which subfields within Behavioral and Cognitive Sciences are more significantly affected by NSF funding?

• National Science Foundation: downloaded award records from 2011 to 2020 from the official website

SCRAPERS

5

MACS 30122

Data Collection

• Google Scholar: scraped academic backgrounds and achievements based on names and emails of award receivers

• Academic Journal Sites: dynamically scraped abstracts of papers published within a three-year period both before and after the funding based on Google Scholar pages

Descriptive Statistics

SCRAPERS

6

Table 1:Descriptive Statistics

MACS 30122

Award Distribution by Amount

SCRAPERS

7

MACS 30122

Figure 1 displays the distribution of NSF funding award amounts, which is observed to be highly skewed. Therefore, in subsequent regressions, we have log-transformed the award amounts to address this skewness.

Figure 1 Distribution of Award Amount

Average Award Amount by Year

SCRAPERS

8

MACS 30122

Figure 2 presents the trend of cumulative NSF funding award amounts over time. It is evident that there is a clear upward trend in funding as the years progress. This observation justifies our model specification that incorporates controlling for time-fixed effects.

Figure 2 Variation of Award Amount by Year

Scatter Plot

SCRAPERS

9

MACS 30122

Figures 3 and 4 respectively depict scatter plots of NSF funding award amounts against the quantity and quality of publications, indicating a positive correlation. However, whether this relationship changes upon the inclusion of control variables requires further determination through regression analysis.

Figure 3 Scatter Plot (funding and pub quantity)

Figure 4 Scatter Plot (funding and pub quality)

OLS Linear Regression

SCRAPERS

10

MACS 30122

Dependent variables

• number of publications (quantity)
• citations of top three most cited papers (quality)

Independent variables

• natural log of award amount
• award (binary encoding)

​

​

​

Fixed effects

• year

Control variables

• h-index & total citations

OLS Linear Regression (baseline)

SCRAPERS

11

Standard errors in parentheses

^* p < 0.1, ^** p < 0.05, ^*** p < 0.01

Table 2: Baseline Regression

MACS 30122

OLS Linear Regression (panel data)

SCRAPERS

12

Standard errors in parentheses

^* p < 0.1, ^** p < 0.05, ^*** p < 0.01

MACS 30122

Table 3: Regression with Time Fixed Effect Controlled

Explanation and Answer to RQ 1

SCRAPERS

13

MACS 30122

• Table 2 presents the results of the baseline regression, revealing that funding award amounts have a significant positive impact on the quantity of scientific output but exhibit a significant negative influence on the quality of scientific output.

​

• Table 3 displays the results of the regression after controlling for time-fixed effects, specifically in models (1) and (2) from Section 1. It is observed that funding award amounts have a significant positive impact on the quantity of scientific output, yet they do not have a significant effect on the quality of scientific output.

​

• Thus, we are able to address Research Question 1: NSF funding significantly enhances the quantity of academic research output in the Behavioral and Cognitive Sciences. However, NSF funding does not have a significant impact on the quality of academic research output in the Behavioral and Cognitive Sciences.

K-means Clustering with TF-IDF

SCRAPERS

14

MACS 30122

We employed cluster analysis to divide the field of Behavioral and Cognitive Sciences into eight subfields, as detailed in Figure 5. We have identified eight subfields within the Behavioral and Cognitive Sciences: Environmental Studies, Psychology, Cognitive Neuroscience, Phonetics, Human Biology, Linguistics, Cultural Studies, and Archaeology.

Figure 3 Cluster Result

Word Clouds by Cluster

SCRAPERS

15

MACS 30122

Word Clouds by Cluster

SCRAPERS

16

MACS 30122

Word Bar Graphs

SCRAPERS

17

MACS 30122

Word Bar Graphs

SCRAPERS

18

MACS 30122

Answer to RQ2

SCRAPERS

19

MACS 30122

Next, based on the results of our clustering, we can calculate the Average Award Amounts by Cluster for each subfield, and then make comparisons. This allows us to answer RQ 2: Even within the same research division of Behavioral and Cognitive Sciences, there are clear disparities in the funding received by different subfields. Cognitive Neuroscience obtains the most funding, followed by Psychology and Environmental Studies with moderate amounts, and Human Biology, Archaeology, Phonetics, and Cultural Studies receive the least.

Figure 7 Percentage of Different Subfields

Answer to RQ2

SCRAPERS

20

MACS 30122

Figure 8 Average Funding of Different Subfields

Regression Grouped by Subfield (RQ 3)

SCRAPERS

21

MACS 30122

Based on the results of the clustering, we conducted grouped regressions for each subfield. Table 4 presents the grouped results. It can be observed that the funding has a significant positive impact on the number of publications across all groups. The coefficients and significant levels for Cognitive Neuroscience, Human Biology, and Environmental Studies are the largest, indicating that funding has a stronger effect within these three subfields.

Response to Comments

SCRAPERS

22

MACS 30122

Q1 Is that a "real" effect or a measurement effect in RQ1?

A1 This is a measurement effect since this is not a structural model.

Q2 the reason of getting funding might be because of the previous performance

A2 We control sholars’ ability(h-index) to address this problem.

Q3 whether you accounted for the size of the academic discipline when considering awards?

A3 Yes. In Table 5 and 6 in appendix, we control the discipline fixed effect.

Q4 You chose citations of top three most cited papers to represent the quality, would the result change, if we choose another number, like 5?

A4 No. The result is fairly robust.

Q5 How do you measure quality? If it is measured by the number of references, it might be subject to a large time lag/bias - work that got published earlier has a higher chance of getting more references.

A5 We used the citations of the top three most cited papers, taking into account the large time lag/bias. This is why we control for time-fixed effects. As long as we ensure that the quality data for Author A in year X is all sourced from year X, along with the inclusion of time-fixed effects, we can ensure they are comparable.

Conclusion

SCRAPERS

23

MACS 30122

• NSF funding significantly enhances the quantity of academic research output in the Behavioral and Cognitive Sciences. However, NSF funding does not have a significant impact on the quality of academic research output in the Behavioral and Cognitive Sciences.

​

• Even within the same research division of Behavioral and Cognitive Sciences, there are clear disparities in the funding received by different subfields. Linguistics obtains the most funding, followed by Psychology, Cognitive Neuroscience, Phonetics, and Cultural Studies with moderate amounts, and Human Biology, Archaeology, and Environmental Studies receive the least.

​

• NSF funding has a stronger effect within Cognitive Neuroscience, Human Biology, and Environmental Studies than that within Archaeology, Psychology, Linguistics, Phonetics, and Cultural Studies

References

SCRAPERS

24

MACS 30122

Appendix

SCRAPERS

25

MACS 30122

Figure 9 Comparison of Average Yearly Citation

Appendix

SCRAPERS

26

MACS 30122

Figure 10 Research Interests Word Cloud

Appendix

SCRAPERS

27

Standard errors in parentheses

^* p < 0.1, ^** p < 0.05, ^*** p < 0.01

MACS 30122

Table 5: Panel with K-means Clustering Subfields

Appendix

SCRAPERS

28

Standard errors in parentheses

^* p < 0.1, ^** p < 0.05, ^*** p < 0.01

MACS 30122

Table 6: Panel with K-means Clustering Subfields