Tools

Curl Compiler

The CurlCompiler class was created to make it easier to interpret and programmatically run curl commands in the Python environment, using the requests library.

On this page

    #Overview

    The CurlCompiler class was created to make it easier to interpret and programmatically run curl commands in the Python environment, using the requests library. It solves the problem of automating HTTP requests that would normally be made from the terminal, allowing detailed parsing of the curl command to identify the HTTP method, headers, authentication, request body and URL.

    In practice, this class lets developers take any curl command and:

    • get its components in a structured way (method, headers, data),
    • run the HTTP request directly through Python code,
    • print the responses in a readable way,
    • generate equivalent Python snippets for reuse.

    This is especially useful in API testing scenarios, system integration or when the curl command is generated by other tools and needs to be used programmatically.

    #Execution Flow

    1. Object initialization: An instance of the CurlCompiler class is created, and it is possible to set the timeout and whether SSL verification is on or off.
    2. Compile the curl command: When the curl command is passed as a string to the compile method, it is split into tokens to extract the HTTP method, headers, body, authentication and URL.
    3. Compilation validation: The command is validated to ensure that it has a valid URL and HTTP method.
    4. Request execution: Using the execute method, the class compiles the command and sends the HTTP request based on the extracted data.
    5. Handle the response: The HTTP response received can be converted into a dictionary for manipulation via response_to_dict, printed in a formatted way with pretty_print_response or inspected manually.
    6. Python code generation: A Python snippet equivalent to the curl command can be generated with generate_python_code for reuse or documentation.

    #Class Methods Table

    MethodDescription
    __init__Initializes the class with timeout and SSL verification settings
    compileParses the curl command and returns its components in a dictionary
    executeCompiles and runs the curl command, returning the HTTP response
    call_endpointRuns the HTTP request with manually defined parameters
    response_to_dictConverts a requests response into a dictionary for easy manipulation
    pretty_print_responsePrints the HTTP response formatted for reading
    get_domainExtracts the domain from a given URL
    generate_python_codeGenerates Python code equivalent to the curl command

    #Key Architecture Points and Insights

    • The class uses lexical parsing via shlex to correctly handle strings and spaces in the curl command.
    • It implements clear encapsulation: parsing, execution, response handling and code generation are separated into methods.
    • The decision to automatically set the POST method when a body is present in a command that was originally GET is a common practice and makes use easier.
    • It uses the requests library directly for execution, ensuring compatibility and robustness.
    • The compile method covers the main curl options (method, headers, basic auth, data) and is extensible.
    • Python code generation makes it possible to quickly integrate external curl commands without manual rework.

    #Class and Methods Description

    #CurlCompiler Class

    #Description

    Class that lets you interpret curl commands to run HTTP requests via Python, making automation, testing and API integration easier. It fully parses the command to retrieve the method, URL, headers, authentication and data, and runs the call using the requests library. It also allows easy handling and display of the response.

    #Constructor Arguments

    ArgumentTypeDescriptionDefault Value
    timeoutintTime limit in seconds for HTTP requests30
    verify_sslboolDefines whether SSL certificate verification is doneTrue

    #1. __init__

    Description

    Initializes the CurlCompiler object by setting the timeout for HTTP requests and whether SSL verification should be done.

    Arguments

    • timeout (int): maximum time in seconds for the request (default 30).
    • verify_ssl (bool): turns SSL verification on/off (default True).

    Returns

    • Returns no value.

    Raises

    • Not applicable.

    Examples

    client = CurlCompiler(timeout=10, verify_ssl=False)

    #2. compile

    Description

    Receives a curl command (string) and fully parses it to extract the HTTP method, URL, headers, authentication and request body. Returns a structured dictionary with this information ready for consumption.

    Arguments

    • curl_command (str): string containing the full curl command.

    Returns

    • dict: dictionary containing the keys method, url, headers, params, data, json, auth.

    Raises

    • ValueError: if the command does not start with "curl" or does not contain a valid URL/HTTP method.

    Examples

    curl = 'curl -X POST https://api.exemplo.com -H "Content-Type: application/json" -d \'{"key":"value"}\''
    compiled = client.compile(curl)
    print(compiled['method'])  # "POST"
    print(compiled['url'])     # "https://api.exemplo.com"
    print(compiled['json'])    # {"key": "value"}

    #3. execute

    Description

    Compiles the curl command and runs the HTTP request, returning the requests.Response object.

    Arguments

    • curl_command (str): curl command to be run.

    Returns

    • requests.Response: response received from the server.

    Raises

    • Propagates exceptions from compilation and HTTP execution (e.g. ValueError, requests exceptions).

    Examples

    response = client.execute('curl -X GET https://httpbin.org/get')
    print(response.status_code)  # 200
    print(response.text)         # Conteúdo da resposta

    #4. call_endpoint

    Description

    Runs a direct HTTP call using explicit parameters (method, url, headers, body, authentication).

    Arguments

    • method (str): HTTP method ('GET', 'POST', etc.)
    • url (str): endpoint URL
    • headers (dict, optional): HTTP headers
    • data (any, optional): request body in raw/text format
    • json_data (any, optional): request body as serializable JSON
    • auth (tuple, optional): (username, password) pair for basic authentication

    Returns

    • requests.Response: response of the HTTP request.

    Raises

    • Propagates exceptions from requests.

    Examples

    resp = client.call_endpoint(
        method='POST',
        url='https://api.exemplo.com',
        headers={'Content-Type': 'application/json'},
        json_data={'msg': 'oi'}
    )
    print(resp.status_code)

    #5. response_to_dict

    Description

    Converts the requests.Response object into a dictionary containing status, headers, body (JSON or text) and a success flag.

    Arguments

    • response (requests.Response): HTTP response object.

    Returns

    • dict: dictionary containing the keys status_code, headers, body, success.

    Raises

    • Not applicable.

    Examples

    resp_dict = client.response_to_dict(response)
    print(resp_dict['status_code'])  # Exemplo: 200
    print(resp_dict['body'])         # Conteúdo da resposta

    #6. pretty_print_response

    Description

    Prints the details of the HTTP response in a formatted and readable way, including status, headers and body.

    Arguments

    • response (requests.Response): HTTP response to be printed.

    Returns

    • None

    Raises

    • Not applicable.

    Examples

    client.pretty_print_response(response)
    # Status: 200
    # Headers: { ... }
    # Body: { ... }

    #7. get_domain

    Description

    Extracts the domain (host) from a full URL.

    Arguments

    • url (str): URL to extract the domain from.

    Returns

    • str: extracted domain, for example "httpbin.org".

    Raises

    • Not applicable.

    Examples

    domain = client.get_domain("https://httpbin.org/post")
    print(domain)  # httpbin.org

    #8. generate_python_code

    Description

    Generates a Python code snippet using requests that implements the same behavior as the curl command received.

    Arguments

    • curl_command (str): curl command to be converted.

    Returns

    • str: formatted and indented Python code.

    Raises

    • Propagates errors from compiling the curl command.

    Examples

    code = client.generate_python_code('curl -X POST https://api.exemplo.com -H "Content-Type: application/json" -d \'{"key":"value"}\'')
    print(code)

    Expected output example:

    # =========================================================
    # EXEMPLO DE USO
    # =========================================================
    
    if __name__ == "__main__":
    
        curl = '''
        curl -X POST "https://httpbin.org/post" \
        -H "Content-Type: application/json" \
        -H "Authorization: Bearer TOKEN_123" \
        -d '{"message":"hello world","value":123}'
        '''
    
        curl = '''
        curl --location --request POST 'http://localhost:8000/embedding'
        '''
    
        client = CurlCompiler(timeout=15)
    
        # Compilar CURL
        compiled = client.compile(curl)
    
        print("\nCURL COMPILADO:")
        print(json.dumps(compiled, indent=4, ensure_ascii=False))
    
        # Executar
        response = client.execute(curl)
    
        # Mostrar resposta
        client.pretty_print_response(response)
    
        # Gerar código equivalente
        print("\nCÓDIGO PYTHON GERADO:\n")
        print(client.generate_python_code(curl))

    This documentation provides a complete and didactic view of how the CurlCompiler class and its main methods work, enabling developers to use it efficiently for parsing and automated execution of curl commands in Python.

    Source: src/web_services_network/utils/curl_compiler.py

    Esc
    ↑↓navigate Enteropen