Core Evaluation

​

Business Skills Workshop

ABRF Annual Meeting

​

May 7^th, 2023

Boston, MA��

Cassie Sather

Research Administration Manager

Shared Resources

Fred Hutchinson Cancer Center

Core Evaluation

The institution is obligated to be a good steward of its resources

​

Evaluation criteria must be standardized across cores

​

Core Evaluation

SR Oversight Process

Core Evaluation

Evaluation - Data Source

A Case for Shared Resources Administration

• Data must be reliable

​

• One source for who is using what

​

• One source for which grants and contracts have been used for X core and Y service

​

• One source for total # grants billed and total $ billed for core (standard reportable value)

​

• Billing activity should be efficient, regularly scheduled, and standardized (if possible)

​

• Having data visualization expertise in-house is a game changer

​

Core Evaluation

Who? and When?

​

- Core Directors (monthly)

​

- Shared Resources Administration (monthly and annual budgeting)

​

- Faculty Advisory Committee (annually)

​

- Survey (annually)

​

- Full Review of Core (when sunsetting a core is being considered)

​

Core Evaluation

Monthly Evaluation

Core Director, Shared Resources Administration

*Faculty are your core’s partners and allies – pay attention to your user data and talk to heavy users regularly

Core Evaluation

Simulated Data

Financials

(overall SR)

Data Fields Required:

​

-Budget Amount

​

-Expense Amount

​

-Accounting Date

​

-Acctg Category

​

-Core Name

Core Evaluation

Simulated Data

Financials

(individual SR)

Data Fields Required:

​

-Budget Amount

​

-Expense Amount

​

-Accounting Date

​

-Acctg Category

​

-Core Name

​

Core Evaluation

Simulated Data

Financials

Data Fields Required:

​

-Invoice Number

​

-Invoice Amount

​

-Invoice Date

​

-Core Name

​

-Customer Name

​

​

Core Evaluation

Simulated Data

Utilization

Data Fields Required:

​

-Invoice Amount

​

-Customer Name

​

-Customer Dept

​

-Core Name

​

-Grant Info (T/F)

​

Core Evaluation

Capacity

Data Fields Required:

​

-Service Name

​

-Service Quantity

​

-Equipment Name

​

-Equipment daily capacity

​

-Equipment Time per Unit Billed

​

-Labor Time per Unit Billed

​

-Service Date

​

-Core Name

​

-Core FTE count

​

-Labor daily capacity

​

-Invoice Amount

​

Core Evaluation

- Grants & Publications Supported

​

- Service Lines

​

- Training/Outreach

• Collect data from core directors through Smartsheet form

​

• Feed data to dashboard

Core Evaluation

- Grants

- Publications

- Service Lines

- Training & Outreach

Visualize Smartsheet data through dashboard

Core Evaluation

Visualize Smartsheet data through dashboard

- Grants

- Publications

- Service Lines

- Training & Outreach

Core Evaluation

Publication Data

• No matter how nice we ask or post on our websites and email signatures, citing of Shared Resources is significantly underreported

​

• Core leaders can identify publications associated with their resources if those publications are put in front of them

​

Core Evaluation

$75,000

Publications

Data Fields Required:

​

-Grant ID

​

-Sponsor Source

​

-Faculty Name

​

-Invoice Date

​

-Invoice Amount

​

-Core Name

​

-PubMed ID

​

-Journal Name

​

-Publication Title

​

-Publication Date

​

​

Data Sources:

​

Pubmed

​

Grants Billed and Cited

​

User Data

Core Evaluation

Publication Tool

We are currently building our own tool

​

• Leverage PubMed API to pull publications associated with our users
• Tool will allow publications to be reviewed and assigned to core (s) easily
• Initial plan is for monthly review
• Plan is to pilot with our core leaders

Submit

Core Evaluation

Annual Evaluation

Faculty Advisory Committee

Committee Structure and Responsibilities

- Recommend a committee comprised of not less than five (5) and no more than nine (9) faculty

- In some cases, a non-faculty level user may be a committee member

- Faculty are your core’s partners and allies – pay attention to your user data and adjust FAC accordingly

- Act as an advisory body to the core and provide recommendations regarding strategic direction

- Annual individual member 1:1 meeting with core leader

- Annual committee meeting where members will review, prioritize, and rank capital equipment requests

​

The Annual Meeting – Director Responsibilities

- Showcase core achievements!!!

- Communicate the core’s importance to the institution

- Highlight your needs

- Replacing outdated instruments

- Adding capacity (instrumentation, staffing)

- Bringing in new technology/services

- Sunsetting underutilized technology/services

Core Evaluation

Annual Evaluation

Making a Case for User Surveys

• FAC are comprised of faculty only

​

• Without surveys, we would miss individual user level input

​

• Surveys provide an opportunity for individual users to show appreciation

​

• Identify common themes that will improve our SR processes

​

​

Core Evaluation

Core Evaluation

Genomics & Bioinformatics Survey Comments

• More automation and cloud adoption.
• It will be better to simplify the order request form.
• The Genomics/Bioinformatics staff is very knowledgeable, and easy to communicate with when issues come up. Always helpful with troubleshooting new protocols.
• Above comments relate to Image analysis: I don't use other bioinformatics services.
• My experience with genomics is nearly always great. I have found bioinformatics to be understaffed and challenging to get ahold of.
• Often the analysis of data is backed up for weeks.
• Elizabeth down in the sequencing lab is super amazing - truly a knowledgeable member of the core facilities, dedicated, and very kind.
• The Genomics Core is sincerely some of the best staff I have ever worked with. They are incredibly knowledgeable, professional, very kind and go out of their way to ensure your project is done correctly. I rely heavily on their expertise (as I know most do). I do not have enough nice things to say about the Genomics Core - they are superb.
• More info about how the samples are run and qc.
• It would be great if we could get an agilent bioanalyzer for QC of single cell libraries since tapestation is not very reliable.

Core Evaluation

Sunsetting Cores – Have a Policy!

Factors to Consider

Institution’s strategy and priorities

• Core services and technology no longer contribute to institution’s strategy
• Core services and technology are low priority for the institution

Service

• Services are available externally at the same quality at a similar or decreased cost. Data quality and integrity of an outsourced service to be considered.
• Services offered are no longer needed by the research community or have become obsolete
• Services or a subset of services can be combined into another core

User Base 

• Ongoing low or decreasing utilization over a two-year period
• Shrinking user base of Core Facility; core facilities may have a narrow user base if they serve key institutional needs
• Feedback from current users

​

Technology

• Core technology obsolescence

�Finances

• Ongoing and increasing deficits with no viable plan for reducing deficits
• Costs to either upgrade technology or buy new technology prohibitive and with no obvious return of investment
• Costs to maintain equipment far exceed the support received from user fees and institutional subsidy

Core Evaluation

Equipment Maintenance Evaluation

Making a Case for Centralized Management

• Service contracts cost institutions millions of dollars each year

​

• Service contracts are critical for service prioritization, increasing up time

​

• Are there opportunities to streamline these maintenance costs?

- Bundle contracts with vendors for multiple cores

- Multi-year discounts

- Reduce or eliminate coverage on underutilized/low priority instrumentation

​

• We need data…

​

​

Core Evaluation

Equipment Maintenance – Getting the Data

Prototype – currently in final design phase using Sharepoint and PowerApps

QR Sticker linked to data and form

View current contract details

​

Report Contract Event or Equipment Change

- Report Preventive Maintenance

- Report Repairs

- Report Replacements

- Report Trades

​

​

Core Evaluation

This Afternoon @ 4:00pm

• Q&A from this presentation or new questions developed throughout the day

​

• Talk with our dashboard guru and see a live demo of dashboards

​

• Brainstorm other ways to make core evaluation easier

​

Thank you!

​

��

Appendix

​

��

Core Evaluation

Core Evaluation

Publications

Core Evaluation

Cores are important!

When done correctly, they are an institutional focal point

​

Recruiting

- Recruits see their $ go further

- Bring in new technology that a recruit needs (entire research community benefits)

- Include funds in start up package earmarked specifically for use in SRs (we are considering)

Sustained research productivity

Access for every investigator to instrumentation and expertise

Avoid duplication

Best means to stay current on instruments and technologies

​

Core Evaluation

- Utilization

- Capacity

Data Fields Required:

​

-Census Date

​

-Active Cage Count

​

-Cage Allotment

​

-PI Name

​

-Room #

​

-Maximum Cage Count per Room

​

​

Core Evaluation

SWOT Analysis

• Sometimes a worthwhile exercise
• During your monthly evaluation, worth keeping these in mind
• In preparation for FAC annual meeting, worth keeping these in mind