Emails, Drew Linzer, director and chief scientist, Civiqs, July 17 and 19, 2018

On Tue, Jul 17, 2018, 7:31 AM Selby, Gardner wrote:

Good morning. I write on behalf of the Austin American-Statesman and the fact-checking PolitiFact Texas project seeking the poll and full poll results described here. We are reviewing the accuracy of a claim by the Republican Party of Texas that a recent poll shows Beto O’Rourke trailing Ted Cruz by 2 percentage points.

 

Note: We rely on attributable on-the-record information for all our stories.

 

Also, I seek to complete our story today.

 

Thanks,

 

g.

 

Want our fact checks first? Follow us on Twitter.

W. Gardner Selby

Reporter / News

Austin American-Statesman

PolitiFact Texas

From: Drew Linzer Sent: Tuesday, July 17, 2018 9:39 AM

 

Hi,

 

This is Drew Linzer, the director of Civiqs. How can I help you?

On Tue, Jul 17, 2018 at 7:45 AM, Selby, Gardner (CMG-Austin) wrote:

Can you email me, or point out, a full break-out of the poll showing a 2-point Senate race in Texas?

10:36 a.m.

July 17, 2018

Civiqs is an online polling firm that conducts daily tracking surveys on issues and campaigns across the United States. You can read more about our company and our research methodology on our website: https://civiqs.com/ and https://civiqs.com/methodology/.

 

We release many of our survey results publicly, here: https://www.civiqs.com/results

 

Other results are available to subscribers only, including the TX-Sen poll that you mentioned in your email.

 

We have been polling the TX-Sen race daily, since February 16, 2017, and have collected over 12,000 total responses. The survey population is self-identified registered voters currently residing in the state of Texas.

 

The information included in the tweet you linked, and the follow-up tweets here:

 

https://twitter.com/markos/status/1016389087006097413

https://twitter.com/markos/status/1016391788536389632

 

are screenshots taken from our results website, showing the state of the race, according to our surveys, as of July 4, 2018.

 

Is there anything else I can help you with?

 

Thanks for the inquiry,

Drew

On Tue, Jul 17, 2018 at 9:13 AM, Selby, Gardner (CMG-Austin) wrote:

Thanks. I will review.

 

FOLLOW-UP QUERIES:

 

If I read right, the cited margin in the Texas race draws on responses gathered online over more than 16 months and from volunteers self-identifying as Texans and registered voters. I get all of this right?

 

Why the lengthy timeframe? Any strengths/weaknesses to it?

 

Chris Wilson, who polls for Sen. Cruz, told me by phone that the Civiqs sample in Texas appears to have been plus-4 Republican when, he said, it’s wise in Texas to poll at +10 Republican or so. Is Wilson right on both fronts? Why or why not?

 

Does Civiqs have recent short-term poll results for the Texas U.S. Senate race? We’re aware of results compiled here by Real Clear Politics.

 

I think you’re saying the full Civiqs poll and results are only available to subscribers. Correct?

 

Who sponsored (or who is sponsoring) the Texas U.S. Senate poll?

 

Anything else you suggest we consider?

 

Thanks again,

 

G.

11:45 a.m.

July 17, 2018

Hi, I'll reply to these questions in order.

 

1. Responses are from interviews conducted online with individuals in the Civiqs Survey panel. The panel is opt-in. Respondents self-report their home state and voter registration status. Selected panelists are sampled and interviewed daily based upon our research methodology: https://civiqs.com/methodology/

 

2. The lengthy timeframe enables Civiqs to perform long-term daily tracking in a sustainable, methodologically rigorous manner. Traditional polls will conduct large numbers of interviews over a small number of days with lengthy gaps between polls: for example, 500 interviews might be completed over a three day fielding period, but then pollsters will wait weeks before another poll is conducted. Instead, we spread out our interviews across more days, and remain in the field continuously. We then use specialized trendline fitting statistical models to produce our final results. The model is described more fully in the "Daily Tracking Models" section of our methodology statement. The result is a much more refined, and always up-to-date estimate of current public opinion: https://civiqs.com/results.

 

3. The Texas U.S. Senate poll results imply a sample that is 31% Democrat, 36% Republican, 33% Independent. It's true that a more Republican sample would produce a larger lead for Cruz over O'Rourke.

 

4. All of our results are based on the same tracking data and model, updated daily.

 

5. Full results are available on a subscription basis. We release results for selected questions at https://civiqs.com/results.

 

6. The Texas U.S. Senate poll was not commissioned or sponsored by any client. Civiqs polls on over 250 questions of political interest each day. Civiqs fields surveys to our own research panel, with no independent sponsor.

 

Best wishes,

Drew

 

From: Selby, Gardner (CMG-Austin)

Sent: Thursday, July 19, 2018 10:40 AM

 

Hello again. I just called about visiting by phone for our fact-check.

 

We had these questions:

 

There’s still no way you can share with us and our readers the full poll, right? It behooves us to ask. As it is, we’re limited to the posted illustration suggesting the poll reached more than 12,000 people over 17-plus  months. That doesn’t comport with your elaboration below.

 

Over what time period did a certain number (what number?) of respondents reply to the question that delivered the cited result?

 

Given that individuals self-select and identify as Texans and registered voters, there’s a risk they’re not either, right? It’s possible too that respondents aren’t adults. What does Civiqs do to deal with these risks?

 

What’s the science behind the reliability of relying on unverified online volunteers to answer queries?

 

Earlier, you didn’t speak to whether the poll under-sampled Republicans. Did it? Why or why not?

 

Chris Wilson, who conducts polls for Sen. Cruz’s campaign, told us by phone he’s skeptical of the indication that only 3 percent of respondents were undecided so many months before Election Day. Were respondents coaxed to commit to one candidate or the other? Why is the undecided figure so low?

 

When will your clients or the public likely see an updated Civiqs poll?

 

Anything else you’d recommend we review in evaluating what’s posted by Civiqs?

 

As ever, we count on attributable on-the-record information for our stories. We intend to wrap up our review today.

 

Thanks again,

 

g.

 

Want our fact checks first? Follow us on Twitter.

W. Gardner Selby

Reporter / News

Austin American-Statesman

PolitiFact Texas

On Thu, Jul 19, 2018 at 10:01 AM, Selby, Gardner (CMG-Austin) wrote:

I spotted this, fyi.

 

12:07 p.m.

July 19, 2018

Thanks. Here is the corresponding post by Markos Moulitsas: https://www.dailykos.com/stories/2018/3/12/1748087/-Announcing-Civiqs-the-coolest-thing-I-ve-helped-build-since-Daily-Kos

 

And a post I wrote when we first started releasing results in March: https://votamatic.org/announcing-the-launch-of-the-civiqs-results-dashboard/

 

The model we use to analyze the data we collect each day is very similar to what's done here:

 

https://projects.fivethirtyeight.com/trump-approval-ratings/

http://elections.huffingtonpost.com/pollster/trump-job-approval

If you would like to see the sorts of models we use applied in academic scientific research, here is a paper I wrote in 2013: https://votamatic.org/wp-content/uploads/2013/07/Linzer-JASA13.pdf

 

The model is called a "Dynamic Linear Model" or "Bayesian Dynamic Model". I don't have any nontechnical materials I can send you. Some of the research at these links may provide an illustration of how it works:

 

http://ftp.stat.duke.edu/WorkingPapers/11-19.pdf

http://helios.fmi.fi/~lainema/dlm/dlmtut.html

https://users.aalto.fi/~ssarkka/pub/cup_book_online_20131111.pdf (chapter 3)

 

Best wishes,

Drew

 

 

 

 

 

 

 

...