User Review Analysis for Public Food Aid Mobile Apps

User Experiences and Missing Design Opportunities in
Mobile Applications for Government Food Assistance

 My Role

UX Researcher,
Principal Investigator

Timeline

February — July 2023

Research Objectives

How are public values supported or violated in existing SNAP mobile apps?
How might SNAP mobile apps be designed to support public values for beneficiaries?

Method

Design Evaluation Framework

 I evaluated the SNAP mobile apps using the Value-sensitive Design framework [1], consisting of:

Conceptual Investigation

Identifying stakeholders and their values

Empirical Investigation

Exploring how stakeholders perceive and experience technologies and embedded values

Performed secondary research to identify public values ideal to embed in government mobile apps

Conducted content analysis of mobile app reviews and r/foodstamps Reddit posts to understand which public values are supported or violated

Technical Investigation

Analyzing existing features or designing new ones to support values

Mapped each feature into each public value elements and translated insights into design opportunities

[1] Batya Friedman, Peter Kahn, and Alan Borning. 2002. Value sensitive design: Theory and methods. University of Washington technical report 2 (2002), 12.

Conceptual Investigation

Secondary Research Findings

Secondary research revealed four types of public values that government systems should support: 

Quality of Information & Service

Accuracy, timeliness, relevancy, clarity, and detail of information. Transactional functionalities.

e.g., virtually completing, and submitting application forms or accessing archival data

Organizational Efficiency

Cutting processing time and costs

e.g., shared public information, IT enabled contact points, and well-trained public staff

Openness

Transparency and accountability of government systems

e.g., the disclosure of budget and expenses

Responsiveness

The extent to which a public organization interacts with citizens
e.g., online case tracking, ability to make and timely responses to inquiries

Empirical Investigation

Content Analysis Process

1. Collecting app reviews

2. Collecting r/foodstamps posts

Web crawling to collect the top 3,000 posts ranked by the number of votes
Filtered out deleted posts  2966 posts

3. Thematic Data Analysis

4. Mapping Features to Values

Quality of Information & Service:  Balance update and management, SNAP eligible items/stores

Organizational Efficiency: Reduced Need for In-person Visits, Online Document Submission

Openness: Explanation for SNAP eligibility/allocation, Filing an Appeal or Complaints

Responsiveness: Interview/renewal notification, App Support

Empirical & Technical Investigation

Key Findings from Content Analysis 

Value 1: Quality of Information and Service

Feature 1

Balance update and management

Automatic refresh of EBT balances in mobile apps improved efficiency and convenience for SNAP beneficiaries

“I hate when I’m in a rush and needing to find out what my ebt food balance is and I’m unable to find my last receipt...[the app] has saved me a lot of time and frustration” 


- Prov A21

Users expressed relief and reduced anxiety, especially during purchases, by quickly verifying their balance

when I’m shopping I could just open the app and be reassured of my balance so I don’t get that ‘holding up of the line, or awkward stare for it not going through’”


-EE A7

Some apps offered additional budget management tools like next benefit deposit date, transaction histories, and suggested budget plans

However, lack of such information could lead to suboptimal budgetary decisions

a Reddit poster, unaware of the next EBT deposit date, felt uncomfortable spending benefits and restricted food purchases to cheap, low-quality, or limited items. 

Had the poster been informed of the next deposit date, the poster could have averted poor diet diversity and financial stress

Transaction histories were desired for transparency and accuracy, allowing users to cross-verify balances and monitor discrepancies

“the app [could] include SNAP purchase

transactions, in order to verify accuracy of balance shown” 


- NAH A22

Feature 2

Showing SNAP Eligible Items/Stores

Users often seek clarification on where SNAP EBT cards can be used, facing potential embarrassment if denied at certain retailers

Only three apps included this feature, and even their data accuracy was questionable

“Today I went into subway and asked if I could use my snap and they said no. I just need clarification on whether or not I could use my

card on certain restaurants.” 


- R144

Users often seek clarification on where SNAP EBT cards can be used, facing potential embarrassment if denied at certain retailers

Only three apps included this feature, and even their data accuracy was questionable

“Today I went into subway and asked if I could use my snap and they said no. I just need clarification on whether or not I could use my card on certain restaurants.” 


- R144

Apps listing SNAP-eligible items are valued for saving time 

Beneficiaries expressed embarrassment when having to abandon purchases due to items not being SNAP-eligible

“I’ve been freaking myself out about going to the store...I worry it’ll look obvious that I’m worried about affording stuff...and utterly horrified of having to put groceries back

at the register because I can’t afford them. Is there some kind of list I can look at so I can make a plan before I go?”


- R204

Features in apps that help users prepare shopping lists based on SNAP eligibility contribute to maintaining dignity in public settings

 “I could make my [shopping] list based on ad prices or prices I knew well enough to estimate. I knew before I

got to the store if I was within my budget”


- Prov G40

SNAP beneficiaries report feeling embarrassed by cashier attitudes when using EBT cards, highlighting the need for discreet transaction designs 

“The cashiers always make it a point for everyone to know

I’m using pebt, and when I pull my card out they go from friendly to annoyed and judgemental. I get really embarrassed because everyone in line just looks at me” 


- R742

Value 2: Organizational Efficiency

Feature 1

Reduced Need for In-person Visits

SNAP app users appreciate the cost and time savings due to fewer in-person government office visits

Reviews indicate challenges in accessing physical offices, especially for those without personal transport or loss of income from taking time off work, emphasizing the socioeconomic benefits of digital services

“This app...saves me...money...i don’t have to drive myself to the Building...therefore i won’t need to spend gas...that can be used on other necessities” 

- MCB G29


“[take] time off work, which caused loss of income” 

- CP G12

SNAP apps help alleviate feelings of embarrassment and preserve dignity by providing assistance without face-to-face interactions, reducing the psychological strain associated with in-person visits to government offices, bypassing long lines and negative attitudes at government offices

“Using this app is a plus...I don’t...have...to mentally prepare myself...if the worker assigned to [me] has had a bad day” 

- NAH A3


“sincerely "Thanks" you for...helping us maintain our dignity by providing accessible information online”

- YTB A14

Feature 2

Online Document Submission

Traditional communication via mail between government offices and SNAP beneficiaries led to delays, missed notifications, and even termination of benefits

“The current address I’m at is not feasible for me to

get mail from DHS...I’m looking into a UPS PO box but I don’t know if that will work?” 


- R73

Online submission occasionally led to inadvertent benefit cancellations or denials, emphasizing the importance of ensuring correct document uploads in the right format to prevent errors

“The pictures may be clear on your phone but the caseworkers will not get a clear picture on their end, so they will deny your case, saying you didn’t turn in your documents or that they were too blurry. It happened to me 7 times already” 


- YTB A43

Value 3: Openness

Feature 1

Explanation for EBT Denial or Amount

Many Reddit posters expressed confusion and frustration over not receiving clear explanations for their SNAP application denials, highlighting a communication gap and the need for increased transparency throughout the application process

“Someone went over my application with me and said everything looks good and told me that my household is

qualified for food stamps. I logged on last night and saw my application was actually denied and no reason stating why.” 


-R1584

Some applicants received explanations for their ineligibility, but they were too technical or lacked details, making it hard to comprehend the decision-making process

Specific information on denial reasons, such as residency requirements or income calculations, and actionable steps for correction could increase applicants' chances of approval upon reapplication

“They have me listed as ineligible and the reason listed is TECHNICAL_ELIGIBILITY_IN_SANCTION_PERIOD. Does anyone know [what] this means?”

- R295


“I moved to IL a little over a month ago, and just received notice that my application was denied because I don’t meet residency requirements. Do I have to live in the state for

a certain amount of time to receive benefits, or is there something weird going on here?” 

- R1998

Beneficiaries also expressed confusion over sudden changes in benefit amounts, seeking detailed explanations for discrepancies to address concerns about potential complications and ensure the correct utilization of benefits

“there was a deposit of $199...I’m supposed to get

$160...I’m not exactly sure what to do or who I should call about this. Does anyone here have experience with this and what should I do?”


- R95

Feature 2

Filing an Appeal or Complaints

Current SNAP apps lack a feature to request appeals, forcing applicants to call government offices, leading to long waiting periods and suggesting the need for app improvements to support the appeal process

“we got denied for "Too much income"...I’ve tried...calling multiple times...just to get the auto reply...to call back

later”


- R2719

Some beneficiaries reported not receiving necessary assistance from caseworkers when attempting to appeal, with instances of caseworkers discouraging appeals and making negative statements, with reports of hostility, discrimination, and false accusations hindering beneficiaries' ability to navigate the benefit decision-making process and access entitled benefits

Fear of potential retaliation led to reluctance in filing complaints against caseworkers, highlighting the need for measures to improve interactions with caseworkers and protect the rights and dignity of beneficiaries during the appeal or complaint filing process

“yelled at me and told me that what I was asking for would be fraud. . . She said I could ask for a hearing but the ruling wouldn’t be in my favor” 

- R71


“the interviewer was very harsh and hostile towards me...asked very personal questions...said I alarmed the system(?) because i was "too young for food stamps" and that he was worried I could be a victim of human trafficking...I was denied again” 

- R97


“if I request a new caseworker, would they make a note of it somewhere so that people could potentially discriminate against me in that way?” 

- R140

 Value 4: Responsiveness

Feature 1

Interview/Renewal Notification

Many app users expressed frustration over untimely app notifications, specifically related to document renewal submissions or upcoming interviews, leading to benefit denial or cancellation

“the app doesn’t notify you...until a few days later and by that time your case will get rejected. They need to update this app and actually put some care into this. [This] is people's food”


- NAH A7

Concerns were raised in 15 Reddit posts and 14 app reviews regarding missing calls from caseworkers, with reports of failed communication attempts leading to benefit denial

“I was sent a letter to...have my phone interview...I called and called and called and left several messages...and no one got back to me...Now, they’ve denied me, saying I didn’t complete my phone interview and that it was my responsibility”


- R2

Feature 2

App Support

40 app reviews expressed dissatisfaction with the app support service or lack thereof, highlighting the potential for technical glitches or lack of support to lead to benefit delays, denial, or cancellation

Dissatisfaction with the lack of real-time or immediate technical support was evident, with instances of users experiencing account lockouts and delays in resolving technical issues, leading to missed deadlines and loss of benefits

“I don’t like feeling helpless...I’ve been trying to do what I gotta do to not delay the process but the app wants to keep crashing”


- NAH A17

Outsourcing the development of SNAP mobile apps to third-party developers has resulted in insufficient app support, creating a situation where neither developers nor government staff adequately address user technical issues

Developers dismisses issues as not their problem and government employees lacked the necessary expertise to assist effectively

User demands for improved government staff training to better understand app functionality and provide support were noted

“it has no option to speak to a representative about this

app..my county assistance...are just as lost as me” 


- CE A1


I was told it’s not our problem”

(CE G6)

Inefficiency in technical app support has led some beneficiaries to believe it was a deliberate strategy by the government to hinder benefit access, fostering distrust

“Guessing this is the new way of not letting people recertify so they can deny your benefits” 

- NAH A15

Lack of app support contributed to users feeling neglected and marginalized, while a well-maintained app conveyed respect towards beneficiaries, highlighting the ethical implications of effective technical support

 “Too bad the developers...have apparently abandoned the app and those that depended on it”

- EE A25


“This app shows that you guys actually do care about people in need...and you do it with dignity and respect” 

- Prov A15

Empirical & Technical Investigation

Implications from Findings

We identified major features that could support each public values. 

However, the analysis revealed that dignity, encompassing self-image in relation to treatment from others and individual agency, is inadequately represented in current SNAP mobile apps. Current designs of SNAP apps and programs were found to compromise beneficiaries' self-esteem and agency, prompting the need for design implications to uphold their dignity.  

Implication 1

Enhancing Self-Esteem of SNAP Beneficiaries

[1] Lynn Dombrowski, Ellie Harmon, and Sarah Fox. 2016. Social justice-oriented interaction design: Outlining key design strategies and commitments. In Proceedings of the 2016 ACM Conference on Designing Interactive Systems. 656–671. 

[2] Jill P Dimond, Michaelanne Dye, Daphne LaRose, and Amy S Bruckman. 2013. Hollaback! The role of storytelling online in a social movement organization. In Proceedings of the 2013 conference on Computer supported cooperative work. 477–490. 

Implication 2

Enhancing Agency of SNAP Beneficiaries with Contestability

[1] Daniel N Kluttz and Deirdre K Mulligan. 2019. Automated decision support technologies and the legal profession. Berkeley Technology Law Journal 34, 3 (2019), 853–890. 

[2] Henrietta Lyons, Senuri Wijenayake, Tim Miller, and Eduardo Velloso. 2022. What’s the appeal? Perceptions of review processes for algorithmic decisions. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. 1–15.

[3] Gerald S Leventhal. 1976. What should be done with equity theory? New approaches to the study of fairness in social relationships. (1976). 

[4] Jason A Colquitt, Donald E Conlon, Michael J Wesson, Christopher OLH Porter, and K Yee Ng. 2001. Justice at the millennium: a meta-analytic review of 25 years of organizational justice research. Journal of applied psychology 86, 3 (2001), 425. 

 

 

GoodIT2023.pdf