What our $49 live site scan found on our own app

We sell a $49 live site scan. In September we ran it on one of our own apps, the same way we would for a customer. We built the scan but found our own app still had three things to fix.

The app we tested is small. People can sign up, sign in and get emails from us, including password resets. It is built with Next.js (a common framework for building web apps), is hosted on Vercel, and uses Google's Firebase for sign-in. We have left its name out for this article, but everything else here comes from the report.

The scan checked the publicly available information on the app. This was what could be seen from a visitor's browser and the domain's public email settings. It did not sign in, and it did not see our code.

The scan reported no critical findings. There were no secret keys in the browser code that anyone can download and no private files left where a stranger could fetch them.

Three things we needed to fix

1. Anyone could send our users a fake "reset your password" email

Picture one of our users getting this at 7am: "Someone tried to sign in to your account. Reset your password now." It shows our app's email address as the sender, with our name on it and a button. The button goes to a fake sign-in page that keeps whatever they type.

Anyone could have sent that email. Nothing in email itself proves who sent it, so a domain publishes three settings in its DNS records, the public settings anyone can look up for a domain. They let email providers check for themselves:

• SPF is a list of the services allowed to send email as us.
• DKIM is a signature our real email carries. The receiving server checks it against a key published on our domain.
• DMARC tells the receiving server what to do with email that claims to be from us but fails both of those checks.

For example, when an email claiming to be from us arrives at a Gmail address, Google's email server checks whether it came from us. If it didn't, it follows our DMARC setting:

• none: it takes no action, so the email usually reaches our unsuspecting user.
• quarantine: it puts the email in our user's spam folder.
• reject: it refuses the email, so it is never delivered.

Ours was set to none, so the fake email could have reached our users looking like it came from us. Our users already get real password-reset emails from us. A fake one would look like any other email we send.

The fix: change the DMARC setting in two steps. First send failing mail to spam ("quarantine"), then block it ("reject"). Before each step, read a couple of weeks of DMARC reports. These are daily summaries that email providers send back, listing every server that sent email as us and whether it passed. They only arrive if the DMARC setting includes an address to send them to, so add one first if it is missing. Check that everything sending our real email passes. For us, that is Firebase, which sends our sign-in and password-reset emails, and Google Workspace, which runs our own inboxes. Skip that check, and the email that gets blocked could be our own password reset.

What it takes: two small changes to the domain's DNS records, with a couple of weeks of reading reports before each one. It takes about a month from start to finish, so it is the one we started first.

2. A hacked script could read what our users type

Our app loads scripts, code that runs in the visitor's browser, from other companies: PostHog for analytics, Google for sign-in, and Vercel for visitor stats. If one of them were ever hacked, or a bug let someone add their own script to the page, that script could read everything our users type into our forms, passwords included, and send it somewhere else.

A website can give the browser a list of the places its scripts are allowed to come from. The browser then refuses anything else. This is called a Content Security Policy (CSP), and the site sends it as a header: an instruction that travels with every page, which the browser reads before the page itself. Our app had no list, so the browser would run any script that got onto the page. The list cannot stop a script from a company on it, but it blocks one added from anywhere else, and it limits where any script can send what it reads.

The fix: publish the Content Security Policy. Start it in report-only mode (Content-Security-Policy-Report-Only), where the browser blocks nothing but reports every script the policy would have blocked. The reports show in the browser's console, the error log in its developer tools, so we only see them in our own browser. To see what happens for real users, the policy also needs an address to send its reports to. Leave it in report-only mode for a week of real use, sign-up and sign-in included, then switch it on by sending the same policy as Content-Security-Policy.

What it takes: about a week. Every outside service the app uses has to go on the list, and the test week catches any we missed.

3. Our analytics tracked visitors before they agreed

PostHog, our analytics, saves an ID in the visitor's browser (a cookie) as soon as the page opens, so it can follow them around the app. There was no banner asking first. If any of our visitors are in the UK or the EU, the law there says we need their agreement before that kind of tracking starts.

The fix: a cookie banner that holds the analytics until the visitor says yes. Cookiebot, Osano and the free vanilla-cookieconsent all do this.

What it takes: an afternoon. Cookiebot and Osano are consent management platforms (CMPs): hosted services, priced by the size of the site, that show the banner and keep a record of each visitor's answer. vanilla-cookieconsent is a free script we would add to the app ourselves. Whichever we pick, the app's own code changes too: PostHog has to start switched off, and only begin tracking once the visitor says yes.

Four smaller things

• We might not find out when the app breaks for someone. The app uses PostHog to see how people use it. PostHog can also do error monitoring, reporting problems like a sign-up button that breaks in someone's browser. That is a separate switch, called exception autocapture, and the scan cannot see whether it is on. With it off, the error stays in that visitor's browser, where they may see an error message, and we only find out if they tell us. Checking takes minutes in PostHog's settings, under Error tracking. If it is off, turn it on there.
• Some visits might not get the padlock in the browser. The app sends a header called HSTS (HTTP Strict Transport Security). It tells browsers to only ever use the app's secure connection (HTTPS, the padlock) and to remember that for two years. Ours covered the app's main address but not the addresses under it, like help.app.example.com. A first visit to one of those could start without the padlock, and on café Wi-Fi someone else on the network could read or redirect that visit. Adding includeSubDomains to the same header covers those addresses for anyone whose browser has already visited the main address. The change takes minutes, but first check that every address under the domain works over HTTPS: browsers remember the rule for two years, and an address that doesn't will stop loading for anyone who has used the app.
• Links to the app can show up without a picture. Pasted into WhatsApp, Facebook or a text message, a link to the app would show no picture, and little more than the app's name. People are more likely to scroll past a link like that. The fix is a few OpenGraph tags: lines in the page's code that tell those apps which title, description and picture to show. This is a fix that can be done in an afternoon.
• Our Firebase key is in the code, and that part is fine. The scan checks for exposed API keys, the codes an app uses to identify itself to an outside service. The app uses Firebase for user login and does have an API key in the code that anyone's browser downloads. This is expected, as it is how Firebase works: the key tells Firebase which project to use, and it is not a password. The risk is that someone could use it to call Firebase sign-up directly, without going through our site, to create junk accounts or send password-reset emails to our users. Two settings in the Firebase console, Firebase's settings website, limit what a stranger can learn or do with it. Email enumeration protection stops strangers checking whether an email address has an account with us. Authorized domains should list only the app's own domains, so Firebase's sign-in pop-up only works there. Both take a few minutes. App Check is an optional extra: it makes Firebase refuse requests that don't come from our real app, but it needs a code change.

What the app already did right

The scan also listed everything that passed, so we knew where not to spend time:

• No secret keys for paid services like Stripe, OpenAI or AWS in the code anyone can download. Those are the keys a stranger can use to run up a bill.
• No source maps were published. A source map is a file that turns the shrunk-down code a browser downloads back into the code we wrote, so without one our original code is much harder to read.
• No private files were exposed. The .env file holding our passwords and keys, and the .git folder holding the project's history, could not be downloaded.
• Apart from HSTS, covered above, the scan checks five security headers, and four were on. The missing one was the Content Security Policy from finding 2. The four that were on:
  • X-Frame-Options stops other sites showing our pages inside their own, a trick used to get people clicking buttons they can't see.
  • X-Content-Type-Options stops the browser guessing a file's type, so a file can't be run as a script when it was sent as something else.
  • Referrer-Policy limits how much of our page's address is passed on to sites people click through to.
  • Permissions-Policy turns off the camera, microphone and location for our pages.
• Every part of the page loads over the secure connection (HTTPS). The certificate behind the padlock, the proof that the address is ours, had 62 days left before it needed renewing.
• Our real email is signed (DKIM), and our domain lists the services allowed to send as us (SPF). Those two were already in place, so the email fix in finding 1 only needs the DMARC change.
• Analytics is installed, so we can see where visitors come from and where they give up. It still needs the consent banner from finding 3.
• A privacy policy is linked from the page.
• The app's robots.txt, a file at the top of the site, asks search engines to stay out of the app. It is a request, not a lock, and it leaves the legal, help and shared-link pages open to them, so we need to check that none of those shows something private.

What this scan could not tell us

The scan saw what a signed-out visitor sees. Four things were outside it:

• Whether one user can see another user's data. If a signed-in user changes a number in the address bar, do they get someone else's records? That is the most common serious flaw in apps built fast. Finding it takes signing in as two different users, or reading the code.
• What our servers do. The code that runs on our servers, and what is in our database. A mistake there can expose user records even when everything above passes.
• What is behind a share or invite link. The scan does not guess at addresses, so it could not see what one of those opens.
• How our sign-in cookies are protected. These keep someone signed in, and a stolen one lets a stranger act as that user. The app sets them after someone signs in, and the scan does not sign in.

A report with nothing critical meant nothing critical showed up from outside. It did not mean our app is secure.

Getting a scan of your app

Send us your app's address on the scan page, and confirm it is your site or that you have permission to have it scanned. Within 3 business days you get a report like this one, with each finding in plain words and the fix for it. An invoice for $49 comes with it. Only pay the invoice if you think the report was helpful.