Curl how to use

The curl guide to HTTP requests

curl is an awesome tool that lets you create network requests from the command line

Published Oct 06 2018

curl is a a command line tool that allows to transfer data across the network.

It supports lots of protocols out of the box, including HTTP, HTTPS, FTP, FTPS, SFTP, IMAP, SMTP, POP3, and many more.

When it comes to debugging network requests, curl is one of the best tools you can find.

It’s one of those tools that once you know how to use you always get back to. A programmer’s best friend.

It’s universal, it runs on Linux, Mac, Windows. Refer to the official installation guide to install it on your system.

Fun fact: the author and maintainer of curl, swedish, was awarded by the king of Sweden for the contributions that his work (curl and libcurl) did to the computing world.

Let’s dive into some of the commands and operations that you are most likely to want to perform when working with HTTP requests.

Those examples involve working with HTTP, the most popular protocol.

Perform an HTTP GET request

When you perform a request, curl will return the body of the response:

Get the HTTP response headers

By default the response headers are hidden in the output of curl. To show them, use the i option:

Only get the HTTP response headers

Using the I option, you can get only the headers, and not the response body:

Perform an HTTP POST request

The X option lets you change the HTTP method used. By default, GET is used, and it’s the same as writing

You can perform a POST request passing data URL encoded:

In this case, the application/x-www-form-urlencoded Content-Type is sent.

Perform an HTTP POST request sending JSON

Instead of posting data URL-encoded, like in the example above, you might want to send JSON.

In this case you need to explicitly set the Content-Type header, by using the H option:

You can also send a JSON file from your disk:

Perform an HTTP PUT request

Follow a redirect

A redirect response like 301, which specifies the Location response header, can be automatically followed by specifying the L option:

will not follow automatically to the HTTPS version which I set up to redirect to, but this will:

Store the response to a file

Using the o option you can tell curl to save the response to a file:

You can also just save a file by its name on the server, using the O option:

Using HTTP authentication

If a resource requires Basic HTTP Authentication, you can use the u option to pass the user:password values:

Set a different User Agent

Inspecting all the details of the request and the response

Copying any browser network request to a curl command

When inspecting any network request using the Chrome Developer Tools, you have the option to copy that request to a curl request:

15 Tips On How to Use ‘Curl’ Command in Linux

Back in the mid-1990’s when the Internet was still in its infancy, a Swedish programmer named Daniel Stenberg started a project that eventually grew into what we know as curl today.

Initially, he aimed at developing a bot that would download currency exchange rates from a web page periodically and would provide Swedish Kronor equivalents in US dollars to IRC users.

Long story short, the project thrived, adding several protocols and features along the way – and the rest is history. Now let’s dive in with both feet and learn how to use curl to transfer data and more in Linux!

We have put together the following list of 15 curl commands for you.

1. View curl Version

2. Download a File

3. Resume an Interrupted Download

Download File Using Curl Command

4. Download Multiple Files

With the following command you will download info.html and about.html from http://yoursite.com and http://mysite.com, respectively, in one go.

5. Download URLs From a File

If you combine curl with xargs, you can download files from a list of URLs in a file.

Download Multiple Files with Curl

6. Use a Proxy with or without Authentication

If you are behind a proxy server listening on port 8080 at proxy.yourdomain.com, do.

where you can skip -U user:password if your proxy does not require authentication.

7. Query HTTP Headers

HTTP headers allow the remote web server to send additional information about itself along with the actual request. This provides the client with details on how the request is being handled.

To query the HTTP headers from a website, do:

Curl Query HTTP Headers

This information is also available in your browser’s developer tools.

8. Make a POST request with Parameters

The following command will send the firstName and lastName parameters, along with their corresponding values, to https://yourdomain.com/info.php.

You can use this tip to simulate the behavior of a regular HTML form.

9. Download Files from an FTP Server with or without Authentication

If a remote FTP server is expecting connections at ftp://yourftpserver, the following command will download yourfile.tar.gz in the current working directory.

where you can skip -u username:password if the FTP server allows anonymous logins.

10. Upload Files to an FTP server with or without Authentication

To upload a local file named mylocalfile.tar.gz to ftp://yourftpserver using curl, do:

11. Specify User Agent

The user agent is part of the information that is sent along with an HTTP request. This indicates which browser the client used to make the request. Let’s see what our current curl version uses as default, and let’s change it later to “I am a new web browser”:

Curl Check User Agent

12. Store Website Cookies

Want to see which cookies are downloaded to your computer when you browse to https://www.cnn.com? Use the following command to save them to cnncookies.txt. You can then use cat command to view the file.

Curl Store Website Cookies

13. Send Website Cookies

You can use the cookies retrieved in the last tip in subsequent requests to the same site.

14. Modify Name Resolution

If you’re a web developer and want to test a local version of yourdomain.com before pushing it live, you can make curl resolve http://www.yourdomain.com to your localhost like so:

Thus, the query to http://www.yourdomain.com will tell curl to request the site from localhost instead of using DNS or the /etc/hosts file.

15. Limit Download Rate

To prevent curl from hosing your bandwidth, you can limit the download rate to 100 KB/s as follows.

Summary

In this article we have shared a brief history of the origins of curl and explained how to use it through 15 practical examples.

Do you know of any other curl commands that we may have missed in this article? Feel free to share them with our community in the comments! Also, if you have questions feel free to let us know. We look forward to hearing from you!

If You Appreciate What We Do Here On TecMint, You Should Consider:

TecMint is the fastest growing and most trusted community site for any kind of Linux Articles, Guides and Books on the web. Millions of people visit TecMint! to search or browse the thousands of published articles available FREELY to all.

If you like what you are reading, please consider buying us a coffee ( or 2 ) as a token of appreciation.

We are thankful for your never ending support.

What is a cURL Command and How to Use It?

cURL command is an important Linux tool, commonly used for data transfer and connection troubleshooting. cURL is powered by libcurl, which is a free URL transfer library at the client side.

Let’s dive deeper and learn how to use it.

What Is cURL Command?

cURL (Client URL) is a command-line tool, which allows to transfer data to or from a server without user interaction using supported libcurl library. cURL can also be used to troubleshoot connection issues.

Check cURL Version

Just like with any Linux command, before we begin working with cURL, we need to log into our VPS. If you need help, check out this tutorial about SSH.

First, let’s check what version of cURL is available with the following command:

The output will show the cURL version a list of supported protocols. Now we can look at some cURL command examples

Basic cURL Command Syntax

Let’s learn how to use cURL commands. The basic syntax of cURL looks like this:

The simplest use of cURL is to display the contents of a page. The below example will render the home web page of testdomain.com.

This will render the complete source code of the homepage for the domain. If no protocol is specified curl will interpret this to HTTP.

cURL Command File Options

cURL commands can download files from a remote location. You can do it in two different ways:

An example of this is as shown below:

The above command will save this as testfile.tar.gz.

The above command will save this as newtestfile.tar.gz.

If for some reason, the download gets interrupted, you can resume it using cURL. You can do this with the following command:

Using cURL, we can also download multiple files, like shown below:

If you want to download multiple files from multiple URL, list all of them in a file. cURL commands can be combined with xargs to download the different URLs.

For instance, if we have a file allUrls.txt which contains a list of all URLs to be downloaded, then the below example can be used to download all files.

cURL Commands for HTTP

cURL can also be used when there is a proxy server. If you are behind a proxy server listening on port 8090 at sampleproxy.com, download the files as shown below:

In the above example, you can skip -U username:password if the proxy does not require an authentication method.

A typical HTTP request will always contain a header. The HTTP header sends additional information about the remote web server along with the actual request. While through a browser’s developer tools you can check the header information, you can verify it using a cURL command.

Below is an example of how to retrieve header information from a website.

Using cURL, you can make a GET and a POST request. A GET request will be as follows:

A sample of a POST request will be as shown below:

Over here text=Hello is the POST request parameter. This behavior would be similar to HTML forms.

You can also specify multiple HTTP methods in a single cURL command. Do this by using –next option, like this:

This contains a POST request followed by a GET request.

Every HTTP request will have a user agent which is sent as part of the request. This indicates the web browser details of the client. By default, a cURL request contains curl and the version number as the user agent details. A sample output is as shown below:

You can change this default user agent information using the below command:

Now the changed output will be:

cURL for Cookies

cURL commands can be used to check what cookies get downloaded on any URL. So, if you are accessing https://www.samplewebsite.com, then you can output to a file, save the cookies and access them using cat or a VIM editor.

Below is a sample of such command:

Similarly, if you have the cookies in a file, then you can send it to the website. An example of such command is as shown below:

cURL for FTP

cURL command supports FTP! You can use them to download files from a remote server.

In the above command, ftp://sampleftpserver is an FTP server which accepts connections. The username and password can be skipped for anonymous FTP connections. Type in the command and watch the progress bar fill up.

You can upload files as well with the command below:

Again, we can skip the username and password for anonymous FTP connections.

Limiting cURL Output

While using a cURL you can’t know how big the output will be. You can restrict bandwidth, to make sure it’s not throttled by cURL.

The below command restricts the bandwidth to 100K:

Conclusion

cURL is a powerful and widely used command. It’s useful when you’re dependent on the command line. It has several options and supports multiple protocols. That’s a pretty great reason to learn this command!

Remember, if you want to learn some advanced commands, simply refer to the manual that should be in all Unix versions:

We hope this tutorial gave you a good jumping off point for using cURL! How will you use this command? Let us know in the comments!

Как пользоваться curl

Нам часто приходится загружать различные файлы из интернета, например, исполняемые файлы программ, файлы скриптов, архивы с исходниками. Но не всегда это нужно делать через браузер. Во многих ситуациях гораздо проще выполнить все действия через терминал. Поскольку таким образом вы можете автоматизировать процесс. С другой стороны, веб-мастерам время от времени приходится тестировать доступность веб-сайтов, проверять отправляемые и получаемые заголовки и многое другое.

Для решения таких задач и задач подобного круга можно воспользоваться утилитой curl. Она позволяет решить намного более широкий круг задач, среди которых даже имитация действий пользователя на сайте. В этой статье мы рассмотрим как пользоваться curl, что это такое и зачем нужна эта программа.

Что такое curl?

Команда curl

Перед тем как перейти к описанию того как может использоваться команда curl linux, давайте разберем саму утилиту и ее основные опции, которые нам понадобятся. Синтаксис утилиты очень прост:

$ curl опции ссылка

Теперь рассмотрим основные опции:

Это далеко не все параметры curl linux, но здесь перечислено все основное, что вам придется использовать.

Как пользоваться curl?

Мы рассмотрели все, что касается теории работы с утилитой curl, теперь пришло время перейти к практике, и рассмотреть примеры команды curl.

Загрузка файлов

Но тут вас ждет одна неожиданность, все содержимое файла будет отправлено на стандартный вывод. Чтобы записать его в какой-либо файл используйте:

Если загрузка была неожиданно прервана, вы можете ее возобновить:

Если нужно, одной командой можно скачать несколько файлов:

Данная команда скачает файл, только если он был изменен после 21 декабря 2017.

Ограничение скорости

Передача файлов

Или проверим отправку файла по HTTP, для этого существует специальный сервис:

В ответе утилита сообщит где вы можете найти загруженный файл.

Отправка данных POST

Здесь мы передаем формой поле password, с типом обычный текст, точно так же вы можете передать несколько параметров.

Передача и прием куки

Затем можно отправить cookie curl обратно:

Передача и анализ заголовков

Аутентификация curl

Точно так же будет выполняться аутентификация на серверах HTTP.

Использование прокси

Выводы

В этой статье мы рассмотрели как пользоваться curl, зачем нужна эта утилита и основные ее возможности. Несмотря на свою схожесть с wget, они очень сильно отличаются. Команда curl linux предназначена больше для анализа и имитации различных действий на сервере, тогда как wget больше подходит для загрузки файлов и краулинга сайтов.

Linux curl Command Explained with Examples

Home » SysAdmin » Linux curl Command Explained with Examples

Transferring data to and from a server requires tools that support the necessary network protocols. Linux has multiple tools created for this purpose, the most popular being curl and wget.

This tutorial will show you how to use the curl command and provide you with an exhaustive list of the available options.

Note: If you do not have curl installed, install it by typing the following in the terminal:

What Is the curl Command?

curl (short for «Client URL») is a command line tool that enables data transfer over various network protocols. It communicates with a web or application server by specifying a relevant URL and the data that need to be sent or received.

curl is powered by libcurl, a portable client-side URL transfer library. You can use it directly on the command line or include it in a script. The most common use cases for curl are:

curl Syntax

The basic curl syntax is as follows:

The system outputs the HTML contents found on the URL provided after the curl command.

If you specify a URL that leads to a file, you can use curl to download the file to your local system:

The progress bar shows how much of the file has been downloaded so far.

The syntax of URLs that are part of the command depends on the protocol. Multiple URLs that differ in one part are written together using braces:

Alphanumeric series are written with brackets:

While nested sequences are not supported, multiple sequences are allowed:

curl Protocols

curl supports numerous protocols for data transfer. Find the complete list below.

curl Command Options

The list of all the available options is given below.