A Critique of The Pew Study on Payday Loans

There is an important lesson to be gleaned from the recent report on payday lending (PDL) from the Pew Charitable Trust, the second in its series on the short-term credit product: a study is not a substitute for reality and, unlike reality, is subject to multiple forms of bias. The media and policymakers should be cautious about relying too much on Pew’s results as being reflective of reality. As I’ve always said, studies should be used as a windsock, and not as a weather report. The dangers of academia and think-tanks is that they are not actually in the trenches of the world they study, and consequently, truth can be lost – truth evident in granular form that can be overlooked by a study.

I believe Pew’s PDL Survey suffers from some major inaccuracies. In addition, there is a degree of loaded language within the report that suggests an overarching bias. I feel I am on solid foundations in charging the study is biased, and that its authors do not have a clear understanding of the industry or the product.

The bias is evident from the outset. Pew’s entire project is called the “safe small-dollar loan research project”, and presupposes that there are “unsafe” loans. Yet there is no definition provided for “safe” or “unsafe”. Pew’s website’s language parrots the same criticisms heard from PDL opponents, citing “triple-digit rates of interest and significant renewal fees, which can put consumers into harmful cycles of debt. This practice of predatory lending has stifled access to more transparent and safer forms of borrowing.” This assertion is made despite logical and factual counterarguments presented by this author and other sources over the past several years. Indeed, a rigorous unbiased study, “Defining and Detecting Predatory Lending” has provided robust evidence that PDLs are not predatory. Apparently, Pew didn’t get the memo.

In fact, it seems odd Pew thinks there are more transparent forms of borrowing. PDLs provide the simplest amount of paperwork one might imagine for a loan. In many states, the disclosures are written so an eight-grader can understand them. Pew’s own survey reports that six in seven borrowers say terms and conditions are clear.

Pew seems to enter this issue with its mind already made up.

Pew claims that, “The Pew Safe Small Dollar Loans Research Project, a two-year initiative, develops research-based, federal policy recommendations to protect consumers from dangerous forms of short-term loans.” Regrettably, the solution Pew defaults to is one of government intervention, rather than other solutions that would more likely to address the real issues of short-term credit. Government intervention in the payday loan arena has always resulted in consumers and businesses being harmed, and jobs being lost.

So it seems apparent that Pew begins with bias. It is reasonable to expect its study would reflect that bias, and as such, does not disappoint.

Bias Specifics

While Pew appears to have randomly dialed and obtained short-survey answers from over 35,000 participants, it only selected 451 storefront payday borrowers for its full-length survey. This is an exceedingly small sample from a group of 12 million. Readers are not provided with the reasons why these individuals were chosen for the longer survey.
Pew itself does not provide the entire list of questions either, telling readers they are “being held for future release”. Why not release everything right now? It is essential for proper interpretation of the results to know about every last aspect of the study. This harms the study’s credibility.

There is a significant possibility for non-response bias. PDLs carry a significant societal stigma. Having personally spent weeks in storefronts and interviewing borrowers, I can report that borrowers are embarrassed to enter a PDL store. They are ashamed to have to ask for help. Thus, when the survey comes around to asking about PDLs, respondents are more likely to lie and deny PDL usage. This may result in many positive experiences with PDL going unreported in the survey. In addition, the psychological bias of the respondents themselves is likely to be skewed towards speaking about the product negatively. People do not like to admit they’ve used a PDL, and as such would be more likely to criticize the product – “Oh, I just used it because I was desperate” – than report a positive experience.

Non-response bias is evident in Pew’s report. On page 54, Pew states that respondents for the full survey were paid $20 if they’d used a storefront lender, and, “Because of their relative scarcity, online payday loan borrowers were given an incentive of $35”. Both Stephens, Inc – the investment bank whose PDL analyst reports are the most comprehensive – and my own experience in online PDL, suggest that 20% of PDLs are done online. Twenty percent is hardly what anyone would call “relative scarcity” in a PDL study. That’s a fifth of all respondents. That suggests that Pew missed a huge swath of online borrowers in its study.

Other Faults

Pew asks a biased question during the interview: “Which of these statements comes closer to your point of view? 1. Payday loan should be more regulated. 2. Payday loans should not be more regulated”. People will be more likely to want to feel safe than unsafe. In this context, the connotation associated with the word “regulated” is one of safety. Consequently, people would be more likely to answer, “more regulated”.

One of Pew’s conclusions is that, even though borrowers may have the option to convert to an installment plan if they cannot repay in full on the due date, they choose not to. Pew relies strictly on data from Washington State, which is not representative of the entire country. CFSA, the industry’s trade association group, requires its members to offer an installment plan if a client renews four times. Pew did not survey CFSA to enquire how many times this option is chosen. Why not?

Pew also held focus groups of ten people in 3 cities, including New York City. First, New York seems an odd choice considering PDLs have been banned there for many years. Indeed, PDL is virtually non-existent in northeastern states. This raises questions as to who these participants were and where they came from. Pew does not reveal how any focus group member was chosen, thus exposing it to credibility issues regarding that selection process. In addition, given the aforementioned stigma, it is more likely that both the group moderator and fellow borrowers would influence respondent comments. Given Pew’s bias, one might expect subtle or even overt verbal and body language cues from the focus group moderator to suggest his or her own bias. People want to please, and they want to feel important, so the likelihood of truthful statements in these focus groups is questionable. After all, if they were being paid to perform, they would feel an even greater obligation to provide the moderator with the desired result.

There are other anomalies. Pew assumed that if a survey respondent answered with a cel phone that they did not have a landline, and that therefore two people may not be living in that household. In fact, many families that reside in the payday loan demographic are more likely to have jettisoned landline usage to save money. Research guru Nate Silver has examined the significant statistical errors in polling with cel phone respondents. By assigning a cel phone respondent a weight of 1, instead of it possibly being 0.5, there may be a significant weighting error in the results.


Finally, a careful read of the report shows frequent use of loaded language intended to influence the reader. Pew only publishes quotes that criticize the product or, at best, offer backhanded or weak praise. Are we to believe these were the only quotes? How about a transcript of the session so readers can verify these claims?

Even a casual reader of the study cannot miss that the authors seem to delight in mentioning (over and over and over again) that the average borrower “stays indebted for five months on a two week loan”. This goes to the oft-cited, alleged “cycle of debt”. Yet Pew’s data runs contrary to data collected by Texas, Washington and Oklahoma. Indeed, in Texas, the OCCC reported that the mode (most frequent data point) for renewals was 1, not the 7-10 renewals Pew wants readers to infer.

Pew’s math just doesn’t add up, either. Advance America’s loan loss rates are 3%, yet Pew suggests the reason for this is that borrowers pay off an Advance America loan by borrowing from another lender. What this doesn’t account for is that if this borrower continued that process for five months, one of two things would happen: 1) principal default, or 2) principal repayment.

With default, some lender somewhere would eventually be left holding the empty bag. In that case, the loan loss rate across the entire industry would be much greater than 3%. Yet, it isn’t, as far as public company reporting (it’s 405%). Indeed, a rate much greater than 9% would kill the entire industry!

With eventual principal repayment, it seems highly unlikely that a person would willingly put themselves back into debt repeatedly for five months each and every time they faced another financial shortfall. Pew’s own results say most people use a method they previously rejected in order to pay off the loan in full, yet Pew expects us to believe that the next time a financial difficulty arose, people would go back to payday loans instead of using the method that eventually solved their problem the first time? This strains credulity. People simply do not repeat significant financial mistakes, and those respondents who had bad experiences took personal responsibility for their choices, as quoted in the study. They would not repeat that same mistake given their self-awareness.

In the most transparent attempt to smear the product, Pew claims “lenders depend on repeat borrowing in order to be profitable”. Can anyone name a business that does not rely on repeat business to be profitable? I think if everyone in the world went to McDonald’s just once, Ronald McDonald would not be the icon he is today.


I could further discredit Pew’s study, but instead permit me to turn the tables. What if I’m completely wrong, and this is the most scientifically rigorous study ever conducted? It doesn’t change what story the media should report, and what policymakers must take note of.

• There is a need for short-term credit in this country.
• Payday lenders are innovating. Many are moving to installment loan products, removing the necessity to repay a loan in full in two weeks.
• If borrowers actually do not take advantage of installment restructuring, PDLs need to better advertise that option.
• Legislative solutions that restrict credit access have historically harmed consumers, not helped them.
• If any legislation is undertaken, it should focus on mandating personal finance classes for high school seniors and/or college students in every state.
• To the extent that borrowers use the product irresponsibly, it is unfair to those who use it properly (and the lenders) to remove the option.

Be Sociable, Share!