Hello, I'm Bunjo, in this topic I will explain how you can make http(s) requests using "Proxy" in Ruby.
Proxy
A proxy is an intermediate server or service that allows computers, users and other resources on a network to access other network services or resources. Essentially, the proxy server receives a user's request, forwards that request to the target server, receives the response from the target server, and forwards that response to the originating user.
Proxies have many different uses:
Controlling Internet Access: In places such as organizations, businesses, or schools, proxies can be used to block or monitor access to certain websites. This can be used to control employee access to work-related websites or to prevent children from accessing harmful content.
Ensuring Privacy and Security: Proxies can be used to hide or anonymize users' real IP addresses. This is preferred by those who want to maintain privacy while browsing the internet or those who want to bypass regional restrictions. Additionally, some proxies keep users safe on open networks (for example, public Wi-Fi networks) by encrypting their internet traffic and increasing security.
Speeding up Internet Access: Proxies can speed up internet access by locally caching frequently requested resources and using techniques such as data compression. This can reduce bandwidth consumption on a large network and allow users to receive faster responses.
Interference and Network Security: Proxies can block malware, block access to harmful websites, and filter incoming and outgoing data by monitoring network traffic. This can be used to increase network security and help organizations secure information.
Proxies can support many different protocols, the most common being HTTP, HTTPS, FTP and SOCKS. These protocols enable the use of proxies for web traffic, file transfers, and other types of network communications.
Proxies can support many different protocols, the most common being HTTP, HTTPS, FTP and SOCKS. These protocols enable the use of proxies for web traffic, file transfers, and other types of network communications.
Proxy Types
Transparent Proxy Servers
They are among the most preferred because they are free. Its biggest problem is that it cannot fully perform IP hiding. There is no encryption for data traffic either.
Anonymous Proxy Servers
Servers that can be used to hide your IP address can be highly successful in this regard. One of their biggest disadvantages is that they cannot show the same success in encryption. There are free, traffic quota and paid options.
Elite Proxy Servers
It is very successful at hiding your IP address and encrypting your web traffic. It can provide the most reliable and fastest connection between servers. Only trial periods are free to use. Especially companies prefer this server type.
There are two different servers depending on the connection type:
HTTP (Web) Proxy Servers
It is generally used to hide the IP address and provide fast Internet access. It can only be used when visiting websites. It does not allow connection to any protocol other than HTTP protocol. It is the most preferred server type for connecting to blocked sites. Various forms such as anonymous, transparent and elite can be used.
SOCKS Proxy Servers
It is slower to operate than HTTP servers. It supports servers such as SMTP and FTP. It is not suitable for real-time online gaming as it runs slowly. Despite SMTP support, it is not recommended for use for email traffic
Src: Proxy Nedir? Proxy Ne İşe Yarar? | Millenicom Blog
How Does HTTP Proxy Work?
Client Request: The client (usually a web browser) makes an HTTP request to access a website.
The request includes the URL of the target website and the type of request (GET, POST, etc.).
Proxy Server Receives Request: The client sends the request to the specified proxy server. This is an HTTP proxy server specified in the client's network configuration.
The client sends the request to the proxy server, not directly to the target server.
Proxy Server Receives and Processes the Request: The proxy server receives the request and performs the necessary operations to redirect the request to the target website. The proxy server can check the content and headers of the request, authenticate, or cache the content.
Redirect to Target Server: The proxy server makes an HTTP request to forward the request it is processing to the target website (for example, a web server).
This request includes the URL of the target server, the type of request, and any data appended to the request.
Receive Response from Target Server: The target server processes the request to the proxy server and responds to the request.
The response is usually an HTTP response appropriate to the client's request (200 OK, 404 Not Found, etc.).
Proxy Server Receives and Processes the Response: The proxy server receives the response from the target server and processes it as necessary.
It can perform the necessary checks to deliver the response to the client and cache the response if necessary.
Response is Transmitted to the Client: The proxy server receives the response from the target server and creates the HTTP response required to forward it to the client.
This response is part of the target server's response and is sent to the client.
Using HTTP Proxy with NET/HTTP
Ruby:
require 'net/http'
# Proxy
proxy_address = '202.131.65.110'
proxy_port = 80
url = URI::HTTP.build(host: 'example.com', path: '/')
request = Net::HTTP::Get.new(url)
http = Net::HTTP.Proxy(proxy_address, proxy_port).new(url.host, url.port)
http.use_ssl = (url.scheme == 'https')
response = http.request(request)
puts response.bodyrequire 'net/http': At the beginning of the code, it indicates that we will use the net/http library. This library is used to make HTTP requests and process responses.
proxy_address and proxy_port: Defined as the address and port number of the HTTP proxy. This information determines the configuration of the proxy that will be used when making requests through the proxy.
url: The target URL is created using the URI::HTTP.build method. This URL represents an HTTP request to the host example.com.
request: HTTP request is created with Net::HTTP::Get.new(url). This request uses the GET method to retrieve a specific resource at the target URL.
http: The HTTP connection is created with proxy settings. An HTTP connection is created with the statement Net::HTTP.Proxy(proxy_address, proxy_port).new(url.host, url.port). If the URL scheme is HTTPS, use of SSL is enabled with http.use_ssl = true.
response: The HTTP request is sent with http.request(request) over the created HTTP connection. This ensures that the request is sent to the target server via the proxy. Then, the response variable is assigned to the response variable.
puts response.body: The body of the response is printed to the console using response.body. This allows content from the target URL to be printed to the screen.
Free Proxy: Get Unlimited Residential Proxies for $6 | Geonode
If your proxy connection is protected by username and password:
Ruby:
require 'net/http'
proxy_address = '202.131.65.110'
proxy_port = 80
username = 'username'
password = 'password'
url = URI::HTTP.build(host: 'example.com', path: '/')
request = Net::HTTP::Get.new(url)
http = Net::HTTP.Proxy(proxy_address, proxy_port, username, password).new(url.host, url.port)
http.use_ssl = (url.scheme == 'https')
response = http.request(request)
puts response.bodyReview of Proxy Request
