I want to access a URL that requires a username/password.I want to try it curl Access it.Now I'm doing something similar:
curl http://api.somesite.com/test/blah?something=123
I got an error.I think I need to specify a username and password as well as the above commands.
How can I do this?
#1st floor
Same but different grammar
curl http://username:password@api.somesite.com/test/blah?something=123
#2nd floor
You can also send a user name by:
curl -u USERNAME http://server.example
Curl then asks you for your password and it will not be visible on the screen (or do you need to copy/paste commands).
#3rd floor
Use the -u flag to include the user name, and curl prompts for the password:
curl -u username http://example.com
You can also include a password in the command, but your password will appear in the bash history:
curl -u username:password http://example.com
#4th floor
This is safer:
curl --netrc-file my-password-file http://example.com
... because passing a normal user/password string on the command line is a bad idea.
The format of the password file is (press man curl):
machine <example.com> login <username> password <password>
Be careful:
- Machine name must not contain https://or similar name!Host name only.
- "machine", "login" and "password" are just keywords; the actual information is what follows those keywords.
#5th floor
In order for the password to at least not pop up in your.bash_history:
curl -u user:$(cat .password-file) http://example-domain.tld
#6th floor
In short, the safest way is to use environment variables to store/retrieve credentials.Therefore, the curl command is as follows:
curl -Lk -XGET -u "${API_USER}:${API_HASH}" -b cookies.txt -c cookies.txt -- "http://api.somesite.com/test/blah?something=123"
Then, your WWW_Authentication api will be called and the http WWW_Authentication header API_HASH with the Base64 encoding values of API_USER and API_HASH will be used.-Lk simply tells curl to follow http 30x redirection and use unsafe tls processing (that is, ignore ssl errors).Double--just bash's grammatical sugar stop processing command line flag.In addition, -b cookies.txt and-c cookies.txt flags use-b to send cookies and-c to store cookies locally to process cookies.
This manual contains more Authentication Method Example .
#7th floor
If you are using a system with a Gnome Keyring application, the solution to avoid direct password exposure is to use gkeyring.py From Key ring Extract password from:
server=server.example.com file=path/to/my/file user=my_user_name pass=$(gkeyring.py -k login -tnetwork -p user=$user,server=$server -1) curl -u $user:$pass ftps://$server/$file -O
#8th floor
You can use the following commands:
curl -u user-name -p http://www.example.com/path-to-file/file-name.ext > new-file-name.ext
The HTTP password will then be triggered.
Reference resources: http : //www.asempt.com/article/how-use-curl-http-password-protected-site
#9th floor
To safely pass the password in the script (that is, to prevent it from being displayed with ps auxf or the log), you can use the -K-flag (reading the configuration from stdin) and heredoc to execute:
curl --url url -K- <<< "--user user:password"
#10th floor
I have the same requirements for bash (Ubuntu 16.04 LTS), and the commands provided in the answer do not work properly in my case.I have to use:
curl -X POST -F 'username="$USER"' -F 'password="$PASS"' "http://api.somesite.com/test/blah?something=123"
You only need to use double quotes in the -F parameter if you are using variables, so from the command line... -F'username=myuser'... You can.
Related Security Statements: As Mr. Mark Ribau As noted in the comment, this command is Process List Display password in ($PASS variable, expanded)!
#11th floor
curl -X GET -u username:password {{ http://www.example.com/filename.txt }} -O
#12th floor
Other answers also suggest that netrc specify a user name and password based on what I read.Here are some syntax details:
https://ec.haxx.se/zh-CN/curl-netrc.html
Like other answers, I would like to emphasize the need to be aware of the security of this issue.
Although I am not an expert, I find these links insightful:
https://ec.haxx.se/cmdline-passwords.html
To summarize:
Using the encrypted versions of the protocol (HTTPS and HTTP) (FTPS and FTP) can help prevent network leaks.
Using netrc can help prevent command line leaks.
Further, it appears that you can also encrypt netrc files using gpg
https://brandur.org/fragments/gpg-curl
In this way, your credentials are not "at rest" (stored) as plain text.
13th floor
Very simple, please do the following:
curl -X GET/POST/PUT <URL> -u username:password
#14th floor
Usually the CURL command is called
curl https://example.com\?param\=ParamValue -u USERNAME:PASSWORD
If you don't have any passwords or want to skip the command prompt to require a simple password, leave the password section blank.
That is curl https://example.com\?Param\=ParamValue-u USERNAME:
#15th floor
The safest way to prompt for credentials to be passed to curl is to prompt for credentials to be inserted.This happens when the user name is passed as previously suggested (-u USERNAME).
But what if you can't pass your user name this way?For example, the user name may need to be part of the url, and only the password is part of the json payload.
tl; dr: In this case, this is the safe way to use curl:
read -p "Username: " U; read -sp "Password: " P; curl --request POST -d "{\"password\":\"${P}\"}" https://example.com/login/${U}; unset P U
read prompts for a user name and password from the command line and stores the submitted values in two variables that can be referenced in subsequent commands and not set.
I will elaborate on why other solutions are not ideal.
Why are environment variables unsafe
- Since the environment is implicitly available to processes, it is not possible to track access to and public mode of environment variable content (ps-eww)
- Applications typically capture the entire environment and record it for debugging or monitoring (sometimes log files are recorded in plain text on disk, especially after an application crashes)
- Environment variables are passed to child processes (thus violating the principle of minimum privilege)
- Maintaining them is a problem: new engineers don't know where they are or what needs are around them - for example, don't pass them on to subprocesses - because they are not executed or documented.
Why it is not safe to type a command directly on the command line, because your secret will eventually be seen by any other user running ps-aux because it lists the commands submitted for each currently running process.It is also because your key subsequently appears in the bash history (once the shell terminates).
Why is it unsafe to include it in a local file? Strict POSIX access restrictions on this file can mitigate this risk.However, it is still a file on the file system and is not encrypted when static.
16th floor
This is much more demanding than OP, but since this is the best result of securely passing passwords to curl, I've added these solutions here for others who arrive here.
Note: -s arg for read command is not POSIX, so it is not available everywhere, so it will not be used below.We'll use stty-echo and stty echo instead.
Note: If in a function, all bash variables below can be declared locals, not unset.
Note: perl is fairly common on all systems I've tried, because it's dependent on many things, whereas Ruby and python are not, so use perl here.If you can guarantee that ruby / python is executed in it, you can replace the perl command with its equivalent.
Note: Tested in bash 3.2.57 on macOS 10.14.4.Other shells/installations may require some minor translation.
Securely prompt the user to enter (reusable) passwords for curling.This is particularly useful if you need to call curl more than once.
For modern shells, echo is built in (by checking which echo):
url='https://example.com'
printf "Username: "
read username
printf "Password: "
stty -echo # disables echoing user input, POSIX equivalent for 'read -s'
read pass
printf "\n" # we need to move the line ahead
stty echo # re-enable echoing user input
echo ${pass} | sed -e "s/^/-u ${username}:/" | curl --url "${url}" -K-
unset username
unset pass
For older shell s, echo is similar to/bin/echo (you can see any echo of it in the process list):
This version cannot reuse this password. Please lower the password instead.
url='https://example.com'
printf "Username: "
read username
printf "Password: "
stty -echo # disables echoing user input, POSIX equivalent for 'read -s'
perl -e '
my $val=<STDIN>;
chomp $val;
print STDERR "\n"; # we need to move the line ahead, but not send a newline down the pipe
print $val;
' | sed -e "s/^/-u ${username}:/" | curl --url "${url}" -K-
stty echo # re-enable echoing user input
unset username
If you happen to need to temporarily store the password in a file, reuse it for multiple commands before clearing the password (for example, because you are using functions to reuse the code, do not want to duplicate the code, and cannot pass values through echo).(Yes, in this case they don't have functionality in different libraries, so they're a bit cheating; I'm trying to reduce them to the minimum code needed to display them.)
When echo is built-in (this is specially designed because echo is built-in but provided for integrity reasons):
url='https://example.com'
filepath="$(mktemp)" # random path, only readable by current user
printf "Username: "
read username
printf "Password: "
stty -echo # disables echoing user input, POSIX equivalent for 'read -s'
read pass
echo "${pass}" > "${filepath}"
unset pass
printf "\n" # we need to move the line ahead
stty echo # re-enable echoing user input
cat "${filepath}" | sed -e "s/^/-u ${username}:/" | curl --url "${url}" -K-
rm "${filepath}" # don't forget to delete the file when done!!
unset username
When echo is something of the/bin/echo class:
url='https://example.com'
filepath="$(mktemp)" # random path, only readable by current user
printf "Username: "
read username
printf "Password: "
stty -echo # disables echoing user input, POSIX equivalent for 'read -s'
$(perl -e '
my $val=<STDIN>;
chomp $val;
open(my $fh, ">", $ARGV[0]) or die "Could not open file \"$ARGV[0]\" $\!";
print $fh $val;
close $fh;
' "$filepath")
printf "\n" # we need to move the line ahead
stty echo # re-enable echoing user input
cat "${filepath}" | sed -e "s/^/-u ${username}:/" | curl --url "${url}" -K-
rm "${filepath}" # don't forget to delete the file when done!!
unset username
