Hackathon Success Metrics That Matter After Demo Day

What do these terms mean here?

A unique participant is one identified person who actually built during the program. A judged project is a submission scored against a rubric published before the event. Post-program status is where each team stands thirty days later: still building, paused, or dissolved. These three terms carry this whole article.

You are writing the RFP. Or you already signed one, the weekend ran, the photos look great, and your CFO asks what the company got for the spend. If the vendor's answer starts with how many people showed up, you have learned almost nothing about whether the money worked.

Attendance is the easiest number in the industry to produce and the hardest to defend. It measures logistics, not outcomes. The research on what actually happens to hackathon projects after demo day is blunt, and any buyer who reads it will stop treating a full room as evidence of anything.

Key Takeaways - Roughly two thirds of hackathon projects show no continued work after the event ends, so survival, not turnout, is the signal worth buying. - Five metrics hold up under scrutiny. Unique participants. Judged projects. University reach. Judge quality. Post-program status at 30 days. - Attendance figures belong in the report as supporting context, never as the headline result. - Insist on a written report with a defined format before you sign. - For example: imagine two vendors pitch the same weekend. One promises 500 registrations. The other promises a report naming 180 builders and every judge by title. The second number is the one you can defend to your CFO.

Why headcount fails as a measure

The most cited academic work on hackathon sustainability followed projects from public hackathons and found that around 65% showed no continued activity after the event ended, and that only about 3.55% were still active five months later (Nolte, Chounta, and Herbsleb, ACM GROUP 2020). A literature review of hackathon outcomes agrees: results are real but uneven, and rarely measured after the weekend (Medina et al., 2020).

So what does this mean for a buyer? Read the findings together. A packed venue tells you recruitment worked. It says nothing about whether anything was built that survives Monday morning. When most output goes dark by design, turnout cannot tell you whether the program produced value. What does? The five metrics below.

There is also a conversion gap on the way in. Devpost's own published guidance puts registrant-to-active-builder conversion somewhere between 5% and 33% depending on the event (Devpost hackathon handbook). A thousand registrations can mean three hundred builders or fifty. Registrations are intent. Builders are the resource you paid to convene.

None of this means hackathons are a bad buy. It means the measurement has to be specified up front, because nobody volunteers it afterward. Research on corporate hackathons notes plainly that return on investment is hard to establish and depends entirely on objectives being defined before the event runs (Roy and Laskowski, 2017).

The five metrics that survive scrutiny

1. Unique participants, not registrations

Ask for deduplicated individuals who built something during the program. One person who attends in two cities and registers twice is still one participant. Counting method matters as much as the count itself. If a vendor cannot explain their deduplication method, treat their other numbers with suspicion.

Registrations still have a use, but only as a ratio denominator. Registrations divided by unique builders is a recruitment-quality metric. A smaller room where most people built beats a huge room where most people watched.

2. Projects submitted to a published rubric

A project that was not judged against criteria published before the event is a demo, not an outcome. The rubric is what makes the count defensible. Public frameworks like Major League Hacking's judging guidance and university-published rubrics such as the one from Claremont Graduate University give sponsors an external reference point, so the bar was set before anyone wrote a line of code.

Ask for three numbers here: projects submitted, projects judged complete against the rubric, and projects that scored above the quality threshold you agreed on. The third number is the one that predicts whether anything continues.

3. University reach, counted honestly

What counts as reach? Named institutions among actual builders. Not flyers. Not club contacts. In Mexico this matters more than in most markets. Campus communities are where many working developers first touch a company's stack. GitHub's Octoverse put registered developers in Mexico at around 1.9 million, growing about 21% year over year (GitHub Octoverse 2024). A program that can name fifteen institutions it drew builders from has produced a recruiting map. A program that cannot name five has produced an email list.

4. Judge quality

Judge quality is a proxy for what the scores mean. A rubric applied by senior engineers from the sponsoring company produces signal: interview lists, follow-up challenges. A rubric applied by whoever was available produces noise. Ask for the judge roster with roles, and ask whether submissions were anonymized. Both questions take thirty seconds.

5. Post-program status at 30 days

Almost nobody reports this metric, and it addresses the 65% problem directly. Thirty days after demo day, every team answers one question: are you still building? Answers sort into still building, paused, and dissolved. Each team still going becomes a concrete follow-up candidate for hiring, grants, or a pilot. Against the research baseline, even modest continuation stands out.

Metric	What to ask for	What it replaces
Unique participants	Deduplicated builders, counting method stated	Registration totals
Judged projects	Submissions scored against a pre-published rubric	Demo-day photo counts
University reach	Named institutions represented among builders	Flyers and club contacts
Judge quality	Roster with roles, anonymized scoring yes or no	"industry experts judged"
Post-program status	Per-team status at 30 days	Nothing. Usually absent entirely

Where attendance belongs

To be clear, turnout is not worthless. It is evidence about one thing: whether recruitment worked. Check-in totals, session fill rates, and no-show rates tell you whether your recruiting channels reached real people. In our own programs, cumulative check-ins across sessions have served exactly this role, as a supporting row in the operations section of a report rather than its headline. The same discipline applies to press mentions: they prove the story traveled, not that anything got built.

The test is simple. Take any attendance-style figure out of a report. If nothing important remains, the report was measuring logistics. If the five metrics above remain, the report was measuring outcomes.

The report format to demand in the RFP

Specify the deliverable before signing. A defensible post-program report has six sections:

1. Program facts. Dates, cities, venues, budget versus plan, and any overrun explained.
2. Participants. Unique builders with counting method, registration-to-builder ratio, demographic breakdowns you agreed on.
3. Projects. Submitted, judged complete, and above-threshold counts, with the rubric attached.
4. Reach. Named universities, institutions, and communities represented among actual builders.
5. Judges and mentors. Roster with affiliations.
6. 30-day follow-up. Per-team status, with next steps for teams still building.

How do you spot a good vendor fast? Vendors who operate this way send a sample report the same week. Vendors who resist have told you something useful.

Common ways measurement goes wrong

• Registrations reported as participants. Ask for the deduplication method every time.
• Project counts without a rubric. Unjudged demos inflate easily and predict nothing.
• Survey satisfaction quoted without response counts. An average score means little until you know whether forty people answered or four hundred.
• Press mentions presented as outcomes. They are a communications result, not a building result.
• Follow-up promised but never delivered. Put the 30-day report date in the contract, not the proposal deck.

A short buyer checklist

What do you ask first? Unique participants, not registrations. Then judged projects, not photo counts. Then university reach among actual builders. Then the judge roster. Then 30-day status.

What do you ignore? Raw registrations. Press counts used as proof of building. Survey scores with no response count. Promises of conversion funnels after the fact.

Why this order? Because each item is cheap to verify and hard to fake. A vendor who can answer all five already runs a real program. A vendor who cannot is selling a weekend.

Frequently Asked Questions

How do you measure hackathon success?

With unique participants, projects judged against a rubric published beforehand, university reach among actual builders, judge quality, and per-team status thirty days after the event. Attendance and press sit alongside these as supporting context, never as the headline.

Is hackathon ROI measurable?

Partially, and only if objectives were defined before the event. Academic work shows most projects go dark quickly, so honest ROI looks like continuation counts, hiring conversations opened, and a written report that survives internal audit, not a conversion funnel invented after the fact.

What should a sponsor report include after a hackathon?

Program facts including budget variance, unique participant count with method, judged project counts, named institutional reach, judge roster, and the 30-day team-status follow-up. Demand the format before signing.

Work With Us

Mobil3 designs and runs hackathons and innovation challenges in Mexico and Latin America, and reports every program against the format described above: unique participants, judged projects, university reach, and post-program status. If you are writing a hackathon into an RFP, see our sample report structure. Book time at calendly.com/mobil3/meeting or write to hello@mobil3.xyz.

About This Guide

This guide was written and fact-reviewed by the Mobil3 Editorial Team from direct program operations across Mexico. For questions about this article or our editorial process, contact us at hello@mobil3.xyz or read more about us. For the operator side of running these programs, read how to run a developer hackathon. Related: developer adoption as a program and working with universities in Mexico.