We Tested ROLL Beauty for 19 Hours and Found Nine Bugs

We are currently building ROLL Beauty, an app that lets users lean their smartphone against a wall, sit on a chair, and do a full 360-degree rotation in about 8 seconds. Users can then drag through the recording to review their hair from the front, side, and back—making it easy to point out specific angles to their hairstylist and ask, "Can we fix this part?"

Our team consists of two people: Nao (Developer) and Ryu (Business & Marketing). We are participating in the RevenueCat Shipaton 2026.

Prior to this session, our app worked perfectly inside the iOS Simulator. It processed 36 dummy images flawlessly with zero bugs reported.

So, we decided to spend an entire night actively trying to break it.

Starting at 10 PM and taking a quick nap along the way, we wrapped up around 5 PM the next day. The process was relentless: capture, analyze the metrics, patch the code, and capture again.

By the end, we uncovered 9 bugs. Several of them were the kind of edge cases you could never find by reading source code for hours on end.

While technical details were covered in our previous write-up, this post focuses on why we chose this stress-testing approach and how it actually unfolded.

1. Recruiting Testers Who Donate Their Hair

We reached out to fellow members of Kobe University’s Entrepreneurship Club with just one criteria: people with long hair.

Coincidentally, the group that assembled consisted of members participating in hair donation—people growing out their hair specifically to cut and donate it later.

We intentionally targeted long hair because it represents the absolute worst-case scenario for our app's computer vision logic.

Our early Python-based tests revealed striking variance statistics:

Face height varies by only 6% across different people.
Overall height including hair varies by 105%, and width varies by 49%.

Bounding dimensions that include hair are far too volatile to serve as baseline anchors. If we had tested exclusively on short-haired subjects, this massive discrepancy would have remained invisible. To find where the system breaks, you must test with people who break it.

Testing with multiple subjects was equally crucial. Testing on oneself introduces hidden biases: standing at the same distance, rotating at the same speed, and recording in the same environment. Switching subjects changes standing positions, rotation speeds, and hair volumes, transforming those subtle variances directly into our bug backlog.

2. Intentionally Creating "Worst-Case" Conditions

We explicitly avoided testing in pristine, controlled environments. Software that only runs under optimal studio conditions will inevitably fail in real-world bedrooms or bathrooms. Users don't choose studio lighting; some capture under dim bedroom lamps, others under harsh fluorescent bathroom lights.

Before recording, we listed our predicted failure conditions:

Documenting these scenarios upfront made root-cause isolation significantly faster when bugs appeared. Most of our predictions proved accurate, though two failed in surprising ways.

3. The 7-Step Iteration Loop

Our workflow followed a strict, repeatable cycle:

Select a scenario (e.g., Incandescent indoor lighting + Far distance)
Have the subject perform an 8-second rotation
Import raw data directly to the Mac
Inspect uncropped source frames and raw CSV metric outputs
Formulate hypotheses based on numerical data
Refactor code and rebuild the application
Have the same subject repeat the exact same test condition

Step 4 was our primary turning point. Initially, we only inspected final cropped frames (36 output images) and mistakenly assumed cropping logic was at fault.

After realizing we weren't looking at raw camera frames, we updated our debug builds to output uncropped source frames alongside raw metric CSVs. Separating capture quality from processing quality drastically improved our night's engineering throughput.

4. Late Night Discoveries: Hardcoded Assumptions and Subject Distance

Failing Shoulder Detection

In our initial physical run, subject heads were severely misaligned. Front-facing shots showed excessive torso space, while side profiles clipped subject heads at the frame edge.

The culprit was twofold:

Flawed Shoulder Threshold:We calculated shoulder width as- Face Width × 1.4 × 2 × 1.30(3.64× face width). Human shoulder width typically ranges between- 2.7× and 2.9× face width. Because the threshold condition was mathematically impossible to trigger, the logic repeatedly fell back to selecting the bottom of the person-mask (the bottom of their shirt).
Per-Frame Zoom Scaling:Calculating zoom dynamically per frame caused sudden zoom spikes on side profiles where facial landmark detection was less stable.

We resolved this by enforcing a core constraint: A subject sitting in front of a phone during an 8-second rotation will not change their apparent size by more than 10%. Outlier values exceeding this threshold are discarded as detection noise rather than physical subject movement.

Subject Distance and Image Resolution

We also discovered that distance alone altered image resolution by 2× under identical lighting and hardware conditions.

When subjects stood too far, the cropped region measured only 330px wide, requiring a 3.3× upscale that smoothed dark hair into featureless blocks. Because the framing UI previously only checked whether a face was visible, subjects assumed they were positioned correctly even when standing too far. We added an adaptive distance check that delays the countdown until the subject enters the optimal range.

5. Statistical Pitfalls: The Danger of Medians

When fixing hair clipping issues at frame boundaries, we found that calculating crop boundaries using median values caused hair tips to get cut off in nearly half of all frames.

Testing across 34 runs showed an average out-of-bounds rate of 43.6%, reaching up to 94% in worst-case frames. Switching our bounding box calculations from the median to the 90th percentile reduced the out-of-bounds rate down to 6.9%.

Medians represent average correctness, not total containment.

6. Morning Retractions: Over-Automation and Model Artifacts

Reverting Automated Hair-Length Detection

Following a brief nap, we attempted to automate hair-length detection to remove manual UI selection.

In practice, this failed. While facing forward during setup, long hair hidden behind shoulders remains invisible to the front camera. The measured baseline was systematically underestimated, forcing output frame bounding to default to square 1080×1080 crops.

Furthermore, numerical values for long hair (4.1–4.5) overlapped directly with short hair worn over dark clothing (3.1–3.2). Realizing this couldn't be resolved purely through thresholding, we rolled back the automated feature and restored manual selection.

Model Segmentation Artifacts

We noticed specific frames blowing out into near-white patches. Initial assumptions blamed backlight or exposure shifts.

However, inspecting raw camera frames showed clean exposure without motion blur or backlight.

Further testing revealed:

The corruption occurred deterministically on identical input frames.
Increasing segmentation precision did not resolve the issue.
Failures occurred specifically at precise rear-facing head angles.
Flipping the input image horizontally or vertically fully restored segmentation quality.

Because the underlying pixel distribution shifted, the segmentation model processed the flipped image correctly. We implemented a recovery step: when a frame's mask degrades unexpectedly, we flip the frame, run segmentation, and mirror the result back.

Color Cast vs. Signal Loss

We previously assumed blown-out highlights on blonde hair represented unrecoverable signal loss. Precise pixel inspection revealed that top 0.5% highlight channels averaged RGB(251, 231, 193)—only the Red channel saturated, while Green and Blue retained gradient detail.

The issue was color cast from warm indoor lighting rather than sensor clipping. Additionally, we found our lighting estimation routine calculated per-frame color temperatures but dropped the results without passing them downstream—an oversight easily fixed in code.

7. Dissecting Visual Quality vs. Algorithmic Fixes

Comparing our output against professional 360° hair catalog references highlighted two main quality bottlenecks:

Input Resolution Limits:Native tracking feeds capped at- 1440×1080. Upscaling a 372px crop to 1080px (2.91× expansion) introduced unavoidable softness. Refactoring the video ingestion pipeline weeks before submission posed too high a risk, so structural pipeline changes were postponed.
Background Bleed:Fine strands of hair over dark backgrounds allowed dark pixels to bleed through, creating faint halos. Attempting topology hole-filling failed because dark gaps spanned only ~4 scattered pixels out of 196,608, making them non-topological gaps.

This highlighted a key engineering lesson: Do not solve physical capture problems purely in code. Requiring light backgrounds during capture eliminated the halo effect instantly, saving significant algorithm development time.

Key Takeaways

Document worst-case test matrices beforehand.Explicitly listing edge cases narrows down root causes during live debugging.
Expose raw pipeline telemetry early.Debugging becomes significantly easier when inspecting intermediate camera frames and raw CSV metrics alongside final outputs.
Avoid introducing complex automation during fatigued periods.Structural automation changes carry high regression risk across the entire pipeline.
Combine numerical metrics with manual visual inspection.Aggregate mask scores can mask structural rendering mistakes like inverted orientation transforms.
Test with real human variability early.Simulator passes provide false confidence; physical edge-case testing reveals real-world software limits.

What is the toughest real-world environment you've tested your application in? We'd love to hear your approach in the comments!