Failure Resolution of cURL Failure to Access TLS Website


Most people are tired of using old systems, both software and hardware. But sometimes they have to stick to it, insist on continuing the life of the system, or need a few pillars of incense to pray for the protection of God and Buddha.
Linux is an excellent modular operating system, which benefits from it. When some components are out of date, in most cases, it can upgrade components by downloading source code and compiling them manually, thus ensuring the availability of the system.
In this process, cURL tools are indispensable, especially in many common development platforms, libcurl libraries are used as the basic tools for downloading. For example, PHP/PYTHON/RUST/NPM. When the cURL fails, it directly causes many development tools to upgrade or install dependency packages that cannot continue.

Today, there was a malfunction. A server of some years reported an error when upgrading a security component.

cURL error 35: error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure

Literally, ssl3 made a mistake when shaking hands.
But in fact, if you use a normal device to visit the same site, plus the - v parameter, you can see that the site is actually using TLS encryption:

$ curl -v https://sh.rustup.rs
* Rebuilt URL to: https://sh.rustup.rs/
*   Trying 13.225.103.123...
* TCP_NODELAY set
* Connected to sh.rustup.rs (13.225.103.123) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/cert.pem
  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
    ...(slightly)

CURL mainly relies on openssl in the processing of https, so in fact, simply recompiling cURL does not work. We must also download the new version of openssl and recompile it.

First, the original curl and openssl are uninstalled on the target server, and the basic compiler system is installed:

$ sudo apt-get purge curl libcurl3 libcurl3-gnutls libcurl4-openssl-dev openssl libssl-dev
$ sudo apt-get autoremove -y

$ sudo apt-get install build-essential libtool make

The libtool make r is actually included in build-essential under normal circumstances, but there are still individual distributions that need to be installed separately, but don't worry, components that have been installed will automatically skip, and it won't take much time.
In this process, we should be glad that apt tool does not rely on cURL, otherwise it would be a disaster:)
But then you have to switch to a normal computer to work, because the download of openssl and cURL source code must be through the available download tools, and usually if the version of openssl is too low, even without cURL, the commonly used wget will not work.
The choice of openssl source code is a pit. If it's a new server, of course, you want to use the latest version, and there are few compatibility issues.
But on an old server, the operating system version is also relatively low, so using the latest version is not necessarily good. Often encounter a variety of problems. This process is likely to require you to try and find a version that is as new as possible, but runs without problems.
At present, there are three stable versions available for download on openssl official website: 1.0.2t/1.1.0l/1.1.1d. All three versions already support tls. After testing, I chose 1.1.0l, which works well on this server.
The downloaded source files can be copied to the target server by scp:

$ scp openssl-1.1.0l.tar.gz master@211.100.34.33:~/

Then log on to the target server:

$ tar xvf openssl-1.1.0l.tar.gz
$ cd openssl-1.1.0l
$ ./config
$ make
$ sudo make install
$ sudo echo "/usr/local/ssl/lib" >> /etc/ld.so.conf

Testing after installation can be done through the s_client built-in openssl:

$ openssl s_client -tls1_2 -connect  sh.rustup.rs:443
CONNECTED(00000005)
depth=2 C = US, O = Amazon, CN = Amazon Root CA 1
verify return:1
depth=1 C = US, O = Amazon, OU = Server CA 1B, CN = Amazon
verify return:1
depth=0 CN = sh.rustup.rs
verify return:1
---
Certificate chain
 0 s:CN = sh.rustup.rs
   i:C = US, O = Amazon, OU = Server CA 1B, CN = Amazon
 1 s:C = US, O = Amazon, OU = Server CA 1B, CN = Amazon
   i:C = US, O = Amazon, CN = Amazon Root CA 1
 2 s:C = US, O = Amazon, CN = Amazon Root CA 1
   i:C = US, ST = Arizona, L = Scottsdale, O = "Starfield Technologies, Inc.", CN = Starfield Services Root Certificate Authority - G2
 3 s:C = US, ST = Arizona, L = Scottsdale, O = "Starfield Technologies, Inc.", CN = Starfield Services Root Certificate Authority - G2
   i:C = US, O = "Starfield Technologies, Inc.", OU = Starfield Class 2 Certification Authority
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIFYDCCBEigAwIBAgIQCYiHqMAcId0Jd6fEfXdC8jANBgkqhkiG9w0BAQsFADBG
MQswCQYDVQQGEwJVUzEPMA0GA1UEChMGQW1hem9uMRUwEwYDVQQLEwxTZXJ2ZXIg
Q0EgMUIxDzANBgNVBAMTBkFtYXpvbjAeFw0xOTA1MjkwMDAwMDBaFw0yMDA2Mjkx
MjAwMDBaMBcxFTATBgNVBAMTDHNoLnJ1c3R1cC5yczCCASIwDQYJKoZIhvcNAQEB
BQADggEPADCCAQoCggEBAN7D4NF6MP8rKvehZtNeSH19cqW5DFD+o8S3rywcyWvo
jkDrdbgxrVhCBf6DY9IlVantRrSJr3N+vmd64y13riRhZC/Q4dqL7S6lyqEtOoj+
   ...(slightly)

The public key issued by the server indicates that the compiled version of openssl can work normally and support tls encryption.
Then you can continue compiling cURL s below, otherwise compiling is wasted time and still unusable.

CURLs are usually available in the latest edition and rarely encounter incompatibilities. It is still downloaded from the working computer and copied to the target server after completion. The process is slight.
Then log on to the target server:

$ tar xvf curl-7.66.0.tar.gz
$ cd curl-7.66.0
# The following line indicates the path of the ssl installation, and OpenSSL 1.1.0l is installed in this directory by default compilation.
# Openssl 1.1.1d is installed in / usr/local. Please confirm the installation path according to your version.
# If you confirm that only this version of openssl is currently installed, you only need the -- with-ssl parameter, which is also possible without additional directory parameters.
# But usually, because of the existence of other open ssl-dependent software, there are other versions of libssl, which are not completely deleted.
# At this point, an accurate path must be worked out.
$ ./configure --with-ssl=/usr/local/ssl
$ make
$ sudo make install

At this point, the cURL already supports TLS-encrypted servers, but it will still fail to use:

curl: (60) SSL certificate problem: unable to get local issuer certificate

This is because there is no ssl-encrypted root certificate installed.
The root certificate can be downloaded at https://curl.haxx.se/ca/cacert.pem. It is recommended that it be downloaded on the worker and then copied to the target server.
The root certificate is placed in the openssl configuration directory, usually: / usr/local/ssl/certs / or / etc/ssl/certs /.

After that, it can be used normally.

Keywords: Linux OpenSSL curl SSL sudo

Added by r4ck4 on Mon, 16 Sep 2019 14:41:21 +0300