Website Best Practices
This post discusses practices promoted by checks in ready, a tool that I've made to help developers check for security best practices in their websites. If you're working on a team that's building something served over HTTP you should check it out!
Table of Contents
This table of contents also serves a quick guide / checklist.
- Redirect HTTP traffic to HTTPS
- Cookies should be set securely
- HSTS headers must be returned
- You should support IPv6
- Return a decent Content-Security-Policy
- Permissions-Policy should exist if the response is HTML
- Don't allow your page to be loaded in a frame
- Referrer-Policy should be same-origin or no-referrer
- Multiple nameservers should be configured
- DNS TTL should be longer than you think
- Don't let browsers detect content-type
- Disable developer tooling in your production environment
- Maybe don't use X-XSS-Protection
- The first part of your HTML is critical
- Set up your favicon and you icon icon
- Don't use schemaless URLs in your HTML
- Configure subresource integrity
- Don't use shared CDNs for static assets
- Disable X-DNS-Prefetch-Control
- Let your feeds be consumed by browsers
- Compress responses
- Expect-CT is enabled by default now
- Remove headers that leak information
- Use a secure SSL configuration
- CORP, COOP, COEP, huh?
- Tell browser where to Report-To
- Don't set Expires for Documents
- Do set Cache-Control
- Configure your email securely
- Configure your email securely, even if you don't send email!
- Some well-known files you should return
- Other guides and resources
Redirect HTTP traffic to HTTPS
You should redirect all HTTP requests to HTTPS using an HTTP 301 redirect. This should be performed before the request reaches your application using a reverse proxy sitting in front it. It's safe to include the path requested in the redirect. When configured correctly this means that the user ends up on the page they are expecting.
You can use the Mozilla SSL Configuration Generator to generate a configuration that handles this redirect in a wide variety of server software.
An alternative to redirecting HTTP to HTTPS is to configure your server to not listen to HTTP connections on port 80. This is a good solution for APIs, CDNs and other URLs that a user will never enter manually into a browser.
Just a warning: I have seen some scanning software flag endpoints that return a
4xxresponse for HTTP requests. Those scanners are simply looking for a redirect on that HTTP connection.
Don't get fancy with other, unexpected, response codes.
Cookies should be set securely
Understand the difference between
If you can, set cookies to
Strict but be aware that means cookies will not be sent when requests are initiated by third-party contexts (i.e. someone links a user to your site).
Lax is the default value in all modern browsers but you should explicitly set this to avoid issues with old browsers.
Secure in all cookies to ensure that cookies are only available in HTTPS contexts.
Setting this helps to prevent accidental leakage of cookies over HTTP connections that could be vulnerable to person-in-the-middle attacks.
Finally, you can use the
Path attribute to further restrict cookies.
This ensures that the cookie will only be sent to the parts of your application that need it.
For example, if you set
Path=/admin then the cookie will only be sent when requests are made to the
Remember that cookies can be read and manipulated by the client. Even with these settings a user or a malicious browser extension/proxy can change the contents of a cookie. You should never store sensitive information in cookies.
The best practice is to use a
HttpOnly cookie to store a session id that the server uses to lookup session details from a data store.
Here's an example
Set-Cookie header from an unauthenticated request to Github:
Set-Cookie: _octo=GH1.1.827422477.1670273756; Path=/; Domain=github.com; Expires=Tue, 05 Dec 2023 20:55:56 GMT; Secure; SameSite=Lax
HSTS headers must be returned
The HTTP Strict Transport Security (HSTS) header tells browsers to disallow insecure requests after a successful secure request.
You should return the
Strict-Transport-Security header on all HTTPS requests.
Browsers ignore HSTS headers sent on HTTP requests so you do not need to include this header in your HTTP → HTTPS redirect response.
Since you should not be supporting HTTP requests it's safe and recommended to set the
max-age of the HSTS header to at least 1 year.
If you're comfortable that all subdomains will be served using HTTPS you should add
includeSubDomains to the header.
With Let's Encrypt and others providing free certificates there is really no excuse to not support HTTPS on all subdomains unless you have specific applications that cannot be configured to use HTTPS.
If this is the case consider moving those applications to another domain and using
includeSubDomains for your primary domain names.
Here is an example HSTS header with a one year expiry:
Strict-Transport-Security: max-age=31536000; includeSubdomains
In the wild you will see a wide range of max-age values, with some major websites going significantly beyond the 1 year recommendation.
Fo example: Twitter, Wikipedia, some Google subdomains and Cash App all set
max-age to two or more years.
For reference, a value of "63072000" represents two years.
If you are confident that you will never need to support insecure requests then you can add the
preload directive and follow the instructions on hstspreload.org to be added to the Chrome preload list.
This is a list of sites that are hardcoded into Chrome (and other browsers) as being HTTPS only.
If you use the Mozilla SSL Configuration Generator this header will be included in your configuration.
You should support IPv6
You should add a
AAAA DNS record that points to the IPv6 address of your web server.
Make sure that you configure your web server to listen on IPv6.
If an ISP or mobile carrier does not assign an IPv4 address to each individual customer they will often assign public IPv6 addresses. Connections to websites that only support IPv4 will be sent via a IPv4 in IPv6 tunnel. Supporting IPv6 will give these users, which includes a large percentage of US cellphones, a more optimal experience.
A handy tool for testing IPv6 support is screenshots.page. It allows you to request screenshots of your website from both an IPv4 and IPv6 connection. You should expect the images to be identical.
readysimply checks for the existence of a
AAAArecord. This doesn't guarantee that your IPv6 version is working correctly.
Return a decent Content-Security-Policy
A Content Security Policy tells modern browsers where it can load resources from and make connections to.
If you are serving a HTML response you must include
Content-Security-Policy in your response headers or a
<meta http-equiv> tag.
You should favour the header unless your page is being served by a hosting provider that does not allow modification of the HTTP headers.
The policy is made up of directives that, with a few exceptions, allow you to specify server origins for different resource types. Before writing a policy for your site you should read through MDN's Content Security Policy introduction.
All Content Security Policies should set the
Ideally this will be set to
none, with explicit definitions for all other relevant directives.
To simplify your policy this might be set to
self to allow resources to be loaded from the same domain.
default-srcto be set to a wildcard value such as
To prevent cross site scripting attacks either
script-src must be set.
connect-src should be configured.
You must not include
This applies to all sites but is particularly important if you allow user generated content.
You should not use
style-src but this has less security implications.
In addition to the unsafe values, you should avoid values that are effectively wildcards.
https:, which was intended to force all assets to be loaded over HTTPS.
That's nice in theory but it allows resources to be loaded from all HTTPS servers which is unlikely to be the desired effect.
Unless you are using legacy HTML embeds,
object-src 'none' should be set.
This disables the use of those legacy object embed mechanisms and closes potential security holes.
You should set
frame-ancestors 'none' to prevent your page from being loaded in a frame.
This is discussed a little more below.
During development and testing use the
Content-Security-Policy-Report-Only header to log and report failures without actually blocking the failing directives.
You can roll out policy changes all the way to production using this header.
For both the
Content-Security-Policy-Report-Only headers you should include a
report-to directive that matches a group in your
Report-To header (see below).
Use the CSP Evaluator to check your policy and compare it to Github.com's policy to see a real-world example of a complex, but secure,
The evaluator gives you advice and feedback about each of the directives in your policy.
It's also worth checking out is the OWASP Content-Security-Policy Cheatsheet which takes a more security focussed look at this header.
Permissions-Policy should exist if the response is HTML
It's a frustrating spec but there's no reason not to support
Permissions-Policy in your HTTP response.
Permissions-Policy allows you specify an allow list of origins for specific browser features.
The specification includes an explainer document that gives a great overview.
A common way to define a
Permissions-Policy is to define a list of features you know you won't use and specify an empty origin.
For example, to disable use of the camera, microphone and autoplaying video you could use:
New features are expected to be added in the future.
To ensure backwards compatibility this means a specific feature missing from the
Permissions-Policy is allowing that feature to be used.
Keep this in mind when you are updating your policy in the future - you need to keep somewhat up to date with new features that browsers are introducing.
A full list of features supported by different browsers is maintained as part of the spec on Github.
Permissions-Policy will never override client side security prompts for usage of certain features.
Expect that browsers will continue to prompt users before allowing access to the camera, microphone, geolocation and related features.
An example of a directive added to the
Permissions-Policyby a specific browser is
interest-cohort. This was proposed by the Chrome team to give sites the ability to open out of FLoC.
Don't allow your page to be loaded in a frame
Unless you have a specific use case that requires it you should never allow your HTML document to be loaded in a frame. Loading the page in a frame (or iframe) opens your up to clickjacking attacks and can give the parent document permissions on your page that you are not expecting. It can also be used to hide the true URL from end users.
There are two different settings that can be used to prevent your page being loaded in a frame:
frame-ancestorsdirective in your
Content-Security-Policythat can be set to
X-Frame-Optionsheader that should be set to
Both of the above allow you to specify URLs that can load your page in a frame if that's required.
Unless you need to support very old browsers you only need to use the
Some security scanning tools will fail you for not including
X-Frame-Optionsin your response headers. Provided you have an adequate Content Security Policy set, and you are not supporting old browsers, you should feel comfortable pushing back.
Referrer-Policy should be same-origin or no-referrer
Referrer-Policy tells browsers what information it should send when making HTTP requests that originate from your page.
Setting this to
no-referrer will tell the browser to not set the
same-origin will set the
Referer header when making requests to the same domain.
Note that some Cross Site Request Forgery (CSRF) tools require the
Referer header to be set on requests.
This is one way that servers can check that the request is coming from the expected page.
Note that a malicious user can set the
Refererheader to any value they like.
If your site does not use
POST forms, use
Otherwise test whether that's possible and fall back
same-origin if it breaks your CSRF protections.
Multiple nameservers should be configured
When setting up your DNS records you should configure at least two, ideally four, nameservers. Most DNS providers give you the ability to host your DNS records on different servers using different domain names.
The more traffic your site has the more nameservers you should have configured. This distributes load for DNS lookups across more servers. Effectively it's a form of load balancing.
For maximum availability, an ideal setup for your nameservers should:
- use different domains to cover for DNS issues on a particular domain (i.e. ns1.linode.com, ns2.linode.com is not ideal)
- use different top level domain extensions to cover for outages with a particular extension (see the suggested Amazon AWS Route 53 setup)
- use servers in different data centres for cover for outages in a particular location (use traceroute to determine if the server is in the same location)
- use servers from different providers
You can use
dig ns <domain> to check the nameservers for a domain.
An example response from brntn.me looks like:
;; ANSWER SECTION:
brntn.me. 172800 IN NS ns-1391.awsdns-45.org.
brntn.me. 172800 IN NS ns-1725.awsdns-23.co.uk.
brntn.me. 172800 IN NS ns-386.awsdns-48.com.
brntn.me. 172800 IN NS ns-574.awsdns-07.net.
The following response for github.com includes a total of 8 name servers from two different providers:
;; ANSWER SECTION:
github.com. 900 IN NS dns1.p08.nsone.net.
github.com. 900 IN NS dns2.p08.nsone.net.
github.com. 900 IN NS dns3.p08.nsone.net.
github.com. 900 IN NS dns4.p08.nsone.net.
github.com. 900 IN NS ns-1283.awsdns-32.org.
github.com. 900 IN NS ns-1707.awsdns-21.co.uk.
github.com. 900 IN NS ns-421.awsdns-52.com.
github.com. 900 IN NS ns-520.awsdns-01.net.
DNS TTL should be longer than you think
Higher TTL values for DNS records reduce the frequency of DNS requests that clients have to make. Removing the DNS lookup from the loading time will improve performance for returning users. If it's unlikely that you will change your DNS records, consider using a long TTL (around 86400, or 24 hours).
If you set a long TTL you need to add an additional coordination step to any DNS-related changes. Start by reducing the TTL significantly, then waiting for the old TTL to expire before making any updates.
Don't let browsers detect content-type
X-Content-Type-Options header to disable browsers auto-detecting content types by setting it to
This means that your server needs to return a valid
Setting this to
nostiff also has a positive affect on the way Cross Origin Read Blocking (CORB) works.
You can read more about that in Google's CORB explainer.
Disable developer tooling in your production environment
It is important to ensure that any developer tooling in your production environment is disabled. This will help to protect the integrity of your production environment and, in some cases, improve performance.
Check that debuggers, profilers, source maps, or any other tooling that would be used to inspect or modify the running code are disabled. Leaving these on can cause performance issues, requests timing out (i.e. with an interactive debug session) and potential security issues.
Unless you are deliberately enabling them, you should check that API discoverability tools like Swagger and GraphQL Introspection are turned off for your production environments. Don't give a potential attacker free information about how your APIs work.
If you are using a framework consult the deployment documentation and make sure that's you're follow all of the production readiness steps.
You should add automated pre-deployment checks to ensure that your production configuration is correct.
For a Python project that might be a simple search for
ipdb, if you are using Swagger that could be a smoke test that ensures
/swagger/index.html is a 404.
Maybe don't use X-XSS-Protection
X-XSS-Protection header can be used to turn on cross-site scripting attack filtering in some older browsers.
Modern browsers, including Chrome and Firefox, do not use this feature, effectively enabling the functionality by default.
Instead you should use a
Content-Security-Policy with a
If you must support older browsers, include this header with the value
1; mode=block to enable the most strict filtering.
Some scanning software will complain when this header is missing. As long as you have an appropriate Content Security Policy, and don't support older browsers, you should feel comfortable pushing back.
The first part of your HTML is critical
The first few hundred bytes of HTML is critical to your page rendering time. Specifically the character encoding must be set in the first 1024 bytes of your HTML document.
Your HTML response must start with a valid doctype.
For almost all modern web pages this will be
<html> tag should include a
lang attribute, indicating the language of the page.
This is specified a
<html lang="en"> for an English language page.
The first tag in your
<head> you should be
<meta charset="utf-8"> (replacing
utf-8 with the appropriate charset).
This ensures that the browser is correctly interpreting your document.
Explicitly configuring these settings prevents browsers from detecting the values and helps to ensure that pages are rendered as quickly as possible.
You can see the HTML5 Boilerplate documentation for more details about the way the
<head> is parsed by modern browsers.
Use Google's Pagespeed Insights to test your rendering performance and see tips for improvements.
Set up your favicon and your icon icon
Follow Audrey Feldroy's excellent Favicon Cheat Sheet and return a ICO file at
In addition you should return a higher resolution PNG or ICO version of the icon in the
<link rel="shortcut icon" href=""> tag in your
Don't use schemaless URLs in your HTML
A schemaless URL starts with
// instead of
When used in tags like
<img> this tells the browser to load the asset with the same scheme as the parent document.
Since we are always loading pages of HTTPS it's best to be explicit and always use
https:// for resource references will mean that pages loaded over HTTP will have mixed content warnings in all modern browsers.
Prevent that by redirecting HTTP traffic to HTTPS with a 301 status code without returning the actual HTML document.
Configure subresource integrity
Subresource Integrety (SRI) prevents resources from being manipulated unexpectedly.
A hash of the file is included in the
integrity attribute of
This feature is available in all modern browsers.
You must use SRI if you are loading assets from a shared CDN where you do not have full control over the file that is being served. Without SRI, changes to the files being served by the CDN will go undetected, creating a significant security risk.
Even for files that you are serving yourself SRI adds an extra layer of security, ensuring that static files are not manipulated in-transit or on-disk. Since this value only needs to be calculated at build time this should have no performance impact.
Don't use shared CDNs for static assets
While the introduction of SRI does help make the use of CDNs for static assets safer, the introduction of HTTP cache partitioning takes away most of the benefits.
As noted above, if you use a CDN for static assets you must use SRI for those resources.
off tells browsers not to make DNS requests for links in a HTML document in advance.
Browsers have started making these DNS requests early in order to offer performance improvements when the links on a page are clicked. Unfortunately this can leak information about the contents of the page to the user's DNS provider.
Privacy advocates do encourage servers to return this header to enhance the privacy of users. This is especially important if your site includes user controlled content.
Since the performance improvement is negligible you should include this header and improve the privacy of your users.
In addition to providing this header, users can configure their browser to disable DNS prefetching if they are concerned about the privacy implications.
Note that this header is non-standard and is not recommended by Mozilla because of inconsistent browser implementations.
Let your feeds be consumed by browsers
RSS and JSON feeds form an integral part of the open web.
You should configure feeds to return the
Access-Control-Allow-Origin header to allow them to be read directly from browser-based applications.
The value of the header can safely be
By default this cross origin resource sharing (CORS) setup will only allow GET requests to the feeds. You can read more about CORS in MDN's introductory document.
All modern servers support
gzip and responses should be compressed if the
Accept-Encoding header includes
HTML compresses extremely well and compressed responses can significantly improve data transfer times. For example, the HTML of my Github dashboard is 33.22kB compressed, and 145.78kB uncompressed. All up the non-image content of the site is 810kB compressed and 3.74MB expanded.
Expect-CT is enabled by default now
Expect-CT header indicated to browsers that they should check certificate transparency logs for the website's certificate.
This is now the default behaviour in all browsers that supported the header (Chromium-based browsers).
You should not include this header in your responses as it is either not supported or not used by browsers previously supporting it.
Some security scanning services might complain about this header if it is missing. Push back. Share the MDN link above explaining that it's the default in all the browsers that supported it.
Remove headers that leak information
Some frameworks, CDNs, servers and gateways include additional headers in the response that can leak information. Generally, you want to avoid including any non-required information in your headers. Any additional information will give an attacker more details about your infrastructure and tooling.
It's especially important to remove any references to specific version numbers. Version numbers can be directly linked to known vulnerabilities, making an attacker's job much easier.
Audit your HTTP response headers by making requests to your site with a tool like Postman or
Look out for headers like:
Use a secure SSL configuration
You should only support
TLSv1.1 for connections to your server.
Older SSL/TLS protocols have known vulnerabilities that make them inappropriate for modern web applications.
The only relevant browsers that do not support this configuration are Internet Explorer 8-10, which you would likely prefer not to support anyway.
Use secure 2048 or 3072 bit RSA keys for your SSL certificates.
If you use
certbot or similar this will be handled for you.
DNS CAA should be configured to ensure that only Certificate Authorities you expect can issue certificates for your domain. The policy should disallow wildcard certificates. This website uses DNS CAA to allow both ZeroSSL and Let's Encrypt to issue certificates using this configuration:
issue: letsencrypt.org flags:0
issue: sectigo.com flags:0
issuewild: ; flags:0
Additionally, those CAA records can include
validationmethods directives to further restrict how ACME clients can issue certificates.
See SSL Lab's SSL and TLS Deployment Best Practices guide for a complete list of practices surrounding HTTPS and certificates.
CORP, COOP, COEP, huh?
Cross Origin Resource Policy (CORP), Cross Origin Opener Policy (COOP) and Cross Origin Embedder Policy (COEP) are part of a set of new HTTP security headers introduced in 2020 / 2021. These headers are supported by all modern browsers and help to prevent both side-channel attacks and cross site scripting attacks.
Scott Helme has the definitive guide on how to use these headers: https://scotthelme.co.uk/coop-and-coep/
Tell browser where to Report-To
Report-To header indicates to browsers how they should report issues to.
This is a relatively new header but is supported by all modern browsers for reporting Content Security Policy issues.
Here is an example
Report-To header from brntn.me:
Here the endpoint is powered by report-uri.com, but it could be a path on your domain configured to handle these reports.
group is the name of the reporting endpoint that can be referenced in other headers.
Report-Toyou should make sure that your Content Security Policy does not include the
report-sampledirective. This directive tells browsers to share the script that violates the policy to your reporting provider. Sending user scripts to your server invites potential privacy issues, especially where those scripts are being used for accessibility purposes.
Don't set Expires for Documents
For dynamic content you should prefer setting the
Cache-Control header over an
Expires allows you to specific a time that a document should no longer be considered valid.
Setting Expires to an invalid date (i.e. "0") will tell a browser or proxy to not cache the document.
The problem with the Expires header is that it relies on the server and client times being accurate. This isn't always going to be reliable and can result in some unexpected behaviour.
A use case where Expires does make sense is for static assets with expiry dates set in the distant future.
Do set Cache-Control
Cache-Control header gives you more control over how and where a document is cached.
Most documents should be able set
HTML documents should use a low
max-age value (less than 24 hours).
Static assets should support cache-busting and use a large
max-age (over 30 days).
private directive to indicate to caches that the response is private and should only be cached in ways compatible with that.
Mark Nottingham provides an in depth guide to caching which all web developers should read.
Configure your email securely
If you send email from your domain you need to ensure that you have SPF, DMARC and DKIM set up. Email administrators should look into MTA-STS and TLS-RPT to ensure that your email is sent via TLS.
A Sender Policy Framework (SPF) DNS record tells mail servers what servers are allowed to send mail for your domain.
This is checked by servers receiving email from your domain.
SPF records were previously implemented as an
SPF DNS record, this has been replaced with a simple
Your SPF record should include
-all to explicitly deny email that is sent from a server that isn't in the allow list.
You should configure your SPF record based on the recommendations of your email provider(s).
You should not return an
SPF DNS record, instead favouring the
Domain Keys Identified Mail (DKIM) allows a domain owner to cryptographically sign parts of a message to ensure that it hasn't been tampered with. Each sending server for your domain will have it's own DKIM DNS record configured. Those records contain a public key which corresponds with the private key used by that sender to sign mail. Like the SPF records, you should configure DKIM based on the recommendations of your email provider(s). A reference to the key is included in the email headers.
The Domain-based Message Authentication, Reporting and Conformance (DMARC) DNS record tells servers what to do with mail that doesn't meet the SPF or DKIM policy.
You should use
p=reject to block mail that fails, and you should include
rua=mailto:<email> to ensure that you receive aggregated notifications of failures.
The DMARC record is served as a TXT DNS record on the "_dmarc." subdomain.
For example, email sent from "brntn.me" will use the "_dmarc.brntn.me" record.
Two guides that are useful to refer to when implementing MX records:
Configure your email securely, even if you don't send email!
If you don't use your domain for sending email you should make sure that mail servers know that. Gov.uk has a detailed guide about how to implement this to help block spoofed emails.
Use a "null MX record" and a deny-all SPF record to prevent email being sent from your domain by others.
The null MX record has a priority of "0" and a value of ".". Most good DNS providers will let you configure this.
An SPF record should be created with the value
This tells mail servers that you do not have any sending servers and to deny all email coming from this domain.
If you want notifications for attempts to send email from your domains you can configure a DMARC record with
Some well-known files you should return
There's two simple text files that all non-trivial websites should return.
The classic is
/robots.txt for telling bots what they can and can't access.
There's no rule that states that bots must follow your guidelines here but most well-behaved bots do.
Google has a good guide on how to create a robots.txt.
Two quick things to note on your robots.txt:
- It's supposed to be accessed by bots. Make sure your firewall isn't blocking access by those user agents.
- It can give an attacker information about your site. Avoid using exact paths for things like admin interfaces, instead use wildcards or use Google's advice about using a
noindexmeta tag on those pages instead.
You should include details on how to contact your security team in a file at
The proposed standard for this file is available.
At a minimum a contact email address and expiry date should be included.
Other guides and resources
- 2023-08-15: Published initial version that has been in progress for 6+ months.
- 2023-08-25: Updates to caching, csp, DNS CAA and Report-To sections based on feedback provided in Github Issues by Seidy.