Sanitize your passwords, please!

Posted on 2022.10.14

I've just stumbled upon a completely weird and obscure error in our Kafka sinks.

2022-10-14T14:19:24.220Z    info    system/logger.go:74     Initializing connection to host: dev-datawarehouse.internal
2022-10-14T14:19:24.220Z    fatal   system/logger.go:112    Connection to Clickhouse failed parse "http://sink:bmVK-3NnDsfSSAGF223b2@Hjn-hn..tZ\n@dev-datawarehouse.internal:8123/default?enable_http_compression=1": net/url: invalid control character in URL
git.dev/sink/system.(*zapLogger).Fatal
/app/sink/system/logger.go:112
git.dev/sink/internals.SetupClickhouseConnection
/app/sink/internals/clickhouse.go:108
git.dev/sink/bootstrap.CreateSink
/app/sink/bootstrap/bootstrap.go:66
main.main
/app/sink/main.go:34
runtime.main
/usr/local/go/src/runtime/proc.go:250

How to approach debugging it? The key phrase is invalid control character in URL. So what are the valid characters in URIs? That StackOverflow answer [1] brings the light to that issue. Allowed characters as defined by RFC 3986 are ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789-._~:/?#[]@!$&'()*+,;=.

So let's see what characters are invalid in our URI:

uri = "http://sink:bmVK-3NnDsfSSAGF223b2@Hjn-hn..tZ\n@dev-datawarehouse.internal:8123/default?enable_http_compression=1"
allowed_chars = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789-._~:/?#[]@!$&'()*+,;="

for char in allowed_chars:
    uri = uri.replace(char, "")

print(uri)
"\n"

Yes, the password contained line break character. Yes, it was generated. You should sanitize your password as usually, password generators are able to generate character sequences that should not be interpreted without sanitization.

[1]	https://stackoverflow.com/a/1547940/4308541