Home>

How does the server know to distinguish between a python-requests request and a Fiddler request?

I send a request to the server using python-requests. The server responds with an error:

{"body":{"email1":{"value":null,"error":"required"}},
"email2":null,"status":400,"htmlencoded":false}

Opening Fiddler, sniffing the same request from requests -exactly the same situation:

{"body":{"email1":{"value":null,"error":"required"}},
"email2":null,"status":400,"htmlencoded":false}

But if in Fiddler itself I click on this request Replay -> Reissue Requests, then the request will be executed again and the response will already be positive and contain a token:

{"body":{"additional":{},
"token":"bSCwMAjwyQ31kRZ3"},
"email":null,"status":200,"htmlencoded":false}

What is the fundamental difference in the method of sending requests between requests and Fiddler that the server reacts so differently? Let me remind you that the request is the same, absolutely, exactly the same.

Here is the query itself:

POST https://account.mail.ru/api/v1/user/signup HTTP/1.1
Host: account.mail.ru
Connection: keep-alive
Content Length: 2567
sec-ch-ua: " Not A;Brand";v="99", "Chromium";v="96", "Google Chrome";v="96"
sec-ch-ua-mobile: ?1
User-Agent: Mozilla/5.0 (Linux; Android 7.1.2; SM-N975F) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.104 Mobile Safari/537.36
Accept: application/json, text/plain, */*
X-Requested-With: XMLHttpRequest
X-Request-Id: 86c3d6b-d649-f6e2-a9f4-968c35b483d5
sec-ch-ua-platform:Android
Origin: https://account.mail.ru
Sec-Fetch-Site: same-origin
Sec-Fetch-Mode: cors
Sec-Fetch-Dest: empty
Referer: https://account.mail.ru/signup?from=main_m_touch
Accept-Encoding: gzip, deflate, br
Accept-Language: ru-RU,ru;q=0.9
Cookies:
Content-Type: multipart/form-data; boundary=deea5d8e1ce0e7b3e38140e462a3979e
--deea5d8e1ce0e7b3e38140e462a3979e
Content-Disposition: form-data; name="extended"
true
--deea5d8e1ce0e7b3e38140e462a3979e
Content-Disposition: form-data; name="more_password_strength"
one
--deea5d8e1ce0e7b3e38140e462a3979e
Content-Disposition: form-data; name="context"
sign up
--deea5d8e1ce0e7b3e38140e462a3979e
Content-Disposition: form-data; name="browser"
{"screen":{"availWidth":"900","availHeight":"500","width":"900","height":"500","colorDepth":"24","pixelDepth": "24","availLeft":"0","availTop":"0"},
"navigator":{"vendorSub":"","productSub":"20030107","vendor":"Google Inc.","maxTouchPoints":"5","doNotTrack":"inaccessible","hardwareConcurrency": "4","cookieEnabled":"true","appCodeName":"Mozilla","appName":"Netscape","appVersion":"5.0 (Linux; Android 7.1.2; SM-N98) AppleWebKit/537.36 ( KHTML, like Gecko) Chrome/96.0.4664.104 Mobile Safari/537.36","platform":"Linux i686","product":"Gecko","userAgent":"Mozilla/5.0 (Linux; Android 7.1.2; SM -N98) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.104 Mobile Safari/537.36","language":"en-US","onLine":"true","webdriver":"false"," pdfViewerEnabled":"false","deviceMemory":"4"},
"flash":{"version":"inaccessible"}}
--deea5d8e1ce0e7b3e38140e462a3979e
Content-Disposition: form-data; name="from"
main_m_touch
--deea5d8e1ce0e7b3e38140e462a3979e
Content-Disposition: form-data; name="sent_me_ads"
true
--deea5d8e1ce0e7b3e38140e462a3979e
Content-Disposition: form-data; name="sent_me_ads_common"
true
--deea5d8e1ce0e7b3e38140e462a3979e
Content-Disposition: form-data; name="name"
{"first": "karina","last": "famina"}
--deea5d8e1ce0e7b3e38140e462a3979e
Content-Disposition: form-data; name="birthday"
{"day": 2,"month": 4,"year": 1996}
--deea5d8e1ce0e7b3e38140e462a3979e
Content-Disposition: form-data; name="sex"
female
--deea5d8e1ce0e7b3e38140e462a3979e
Content-Disposition: form-data; name="login"
sfwergdhj5
--deea5d8e1ce0e7b3e38140e462a3979e
Content-Disposition: form-data; name="domain"
mail.ru
--deea5d8e1ce0e7b3e38140e462a3979e
Content-Disposition: form-data; name="password"
===aaa===aaa
--deea5d8e1ce0e7b3e38140e462a3979eContent-Disposition: form-data; name="restore"{"secret": "-","secret_answer": "-","additional_email": "[email protected]"}
--deea5d8e1ce0e7b3e38140e462a3979e
Content-Disposition: form-data; name="htmlencoded"
false
--deea5d8e1ce0e7b3e38140e462a3979e
Content-Disposition: form-data; name="utm"
{"source": "","medium": "","campaign": "","term": "","content": ""}
--deea5d8e1ce0e7b3e38140e462a3979e--

You can enable logging of the request sent by requests to see what the difference is between requests: github.com/gil9red/SimplePyScripts/blob/… . But perhaps the server does not like your User-Agent from requests, such an assumption. you didn't show your code. And to add a User-Agent, you need to fill in the headers in the request or through the session (github.com/gil9red/SimplePyScripts/blob/…)

gil9red2022-01-23 20:00:47

Yes, no, it is also indicated in the headers, this can be seen in the raw request that I attached. The request is repeated exactly, 1 to 1. This raw is everything that passes requests in conjunction with my code.

Pavel2022-01-23 20:05:51

Prodebagal according to your method. I decided to send a request through Fiddler and sniff it through Charles (funny, I know). And Fiddler also got an error from the server, and got it in the future until I turned off traffic forwarding from Fiddler to Charles. After that requests began to be normally sent again. Requests, in turn, receives errors regardless of whether it is sniffed or not.

Pavel2022-01-23 20:18:52

Question... If the requests are the same, is there a problem with https?

gil9red2022-01-24 06:34:13
python requests fiddler