Solving DKIM verification FAILED with Bad Format in Gmail email messages

DKIM (Domain keys identified Mail) is a scheme for which allows a receiver to verify that the email originated (or authorized) by the domain’s owner via a digital signature.

Having DKIM signature adds credibility to the email messages sent from the origin host/domain, which is crucial for automated emailing system to avoid the messages from being suspected as SPAM email or spoofed email.

I managed to set up DKIM for an academic journal website which I’ve managed. The journal’s runs on Open Journal System web application, the addition of DKIM is crucial to avoid GMail or Microsoft Live from labeling the automated emails sent from the academic journal from being labeled as spam.

At first I found that the DKIM scheme that I’ve setup was running fine and the email messages was verified correctly from my Organizational email domain. However, I’ve noticed a problem when the automated email sent from OJS is not properly verified by Gmail-addressed account (@gmail.com). Upon inspection in the email header, I’ve noticed that the GMail marked the DKIM signature sent from my domain as “bad format”. Example below:

DKIM:	'FAIL' with domain jcrinn.com

dkim=neutral (bad format) header.i=@example.com header.s=mail header.b=AbCdE5g;

After hours of searching and debugging, including referring to the DKIM NS TXT record for reference, I finally found out that Gmail treat the “g=*” optional parameter as required, and thus I’ve to append “;g=*” to the DKIM TXT record on my domains’ DNS record.

So it become similar like this:

TXT default._domainkey  v=DKIM1; p=yourPublicKeywHiCHi5+abit+1OnG; g=*

After altering the records, it seems GMAIL finally able to verify the automated emails sent from my OJS-based web application

The “signed-by” is visible when DKIM is successfully validated by GMail

Hopefully this will work out fine for you too!

P/S: DigitalOcean has an excellent tutorial on DKIM installation and setup in GNU/Linux operating system.

How to convert character encoding in text files

Here is how to convert text files from one character encoding to another in GNU/Linux:

#eg1
iconv -f ASCII -t UTF-8//IGNORE file.txt -o output.txt

#eg 2
iconv -f ISO-8859-1 -t UTF-8//TRANSLIT file.txt output.txt

The -f parameter denotes “from” and -t parameter denotes “to” character set.
//IGNORE means the “iconv” will ignore any characters that are not available in the target character set.

While “//TRANSLIT” means the converter will attempt to substitute characters that are not available in the target character set to the closest characters available, failing that, “???” will be replaced in its place.

Most GNU/Linux distribution have iconv preinstalled, if not, please consult your distribution documentation.

Check if your web server supports Brotli Compression

Brotli is a new compressed data format developed by Google for compressing web data. It is documented in RFC7932. Currently, almost all modern web browser support Brotli which compressed better and faster than Deflate.

Brotli is can be enabled in most popular web server including:

  • Apache HTTPD – through mod_brotli (for release after 2.4.26)
  • Nginx – ngx_brotli (provided by Google)
  • Node.js (trough shrink-ray module
  • LightSpeed (since version 5.2)
  • Microsoft IIS (through IIS-brotli extension, for IIS 7.5 and above)

Testing for Brotli Support

KeyCDN.com has provided a tool for testing whether your website supports Brotli compression.

You can go over the website and get your server tested. For nginx webserver, ngx_brotli will automatically downgrade to gzip if the browser does not support brotli encoding

 

How to Change nginx server signature from source code

This post is rather a note to myself just in case the next time I want to recompile a new nginx server (I recently compiled a new nginx release in order to enable support for the new brotli encoding and http2 protocol)

The exact file that you need to change is :
/nginx-1.13.7/src/http/ngx_http_header_filter_module.c

Here’s a snippet of what you need to change:

static u_char ngx_http_server_string[] = "Server: yourservername" CRLF;
static u_char ngx_http_server_full_string[] = "Server: yourservername/1.0" CRLF;
static u_char ngx_http_server_build_string[] = "Server: " NGINX_VER_BUILD CRLF;

replace NGINX_VER and NGINX_VER_FULL with your own string.

IMPORTANT: Do not redefine NGINX_VER constant!

Do not redefine NGINX_VER constants as it would be used in various installation scripts after compilation. For example, the Let’s Encrypt “certbot” tool is dependent on the factory setting of NGINX_VER constant.

BONUS: Change default error page

You can further confuse various network scanners by changing the nginx built-in default error page. Although you can change 4xx and 5xx error page easily in server configuration. Nmap is known to detect nginx installation by looking into the nginx built-in error page.

In order to prevent nmap from further detecting your webserver version and configuration, you can change the default built-in error page using through this file:

/nginx-1.13.7/src/ngx_http_special_response.c

Look for these lines:

static u_char ngx_http_error_full_tail[] =
"<p>&nbsp;</p><hr><center>Copyright &copy; 2018 Mohammad Hafiz bin Ismail (mypapit at gmail.com )" CRLF
"<br /><small><a href=\"https://blog.mypapit.net\">Mypapit Personal Blog</a></small></center>" CRLF
"</body>" CRLF
"</html>" CRLF
;

 

And change the HTML tags accordingly to suit your need, note that you can remove NGINX_VER_BUILD entirely to hide your NGINX version.

You can also customized the built in HTTP code special response, from this :

static char ngx_http_error_502_page[] =
"<html>" CRLF
"<head><title>502 Bad Gateway</title></head>" CRLF
"<body bgcolor=\"white\">" CRLF
"<center><h1>502 Bad Gateway</h1></center>" CRLF
;

to include the “Viewport” meta-tag in order to support mobile devices:

static char ngx_http_error_502_page[] =
"<html>" CRLF
"<head>" CRLF
"<link href=\"https://fonts.googleapis.com/css?family=Lato|Slabo+27px\" rel=\"stylesheet\" />" CRLF
"<meta name=\"viewport\" content=\"width=device-width, initial-scale=1.0\">" CRLF
"<title>502 Bad Gateway</title></head>" CRLF
"<body>" CRLF
"<h1>502 Bad Gateway</h1>" CRLF
"<p>Somebody just fucked up at our end :(</p>" CRLF
;

Just make sure you test the nginx configuration after compiling before deploying it in production environment.

Further Reading: Compiling nginx

A rather complete nginx compiling guide can be found from these websites:

  1. How to Compile Nginx From Source on Ubuntu 16.04
  2. Install Nginx from source code on Ubuntu 14.04 LTS