curl interface test Chinese parameter garbled problem troubleshooting and thinking

1 Problem phenomenon

curl "https://api.bugpk.com/api/weather?city=深圳"

Interface test using http get method and no data is returned.

2 Troubleshooting process

2.1 Switch the http post mode, return the JSON format data normally, and the server-side interface service is normal.

curl -d "city=深圳" "https://api.bugpk.com/api/weather"

Return the json data slightly.

2.2 Turn on the detailed mode

curl -v "https://api.bugpk.com/api/weather?city=深圳"

Found that the Chinese in the request line has become garbled:

> GET /api/weather?city=娣卞混 HTTP/1.1

2.3 Try switching cmd encoding

chcp 65001    # switch to UTF-8

Still failing, still garbled in the request.

2.4 Compare Testing: Use the 'httpbing.org' test

C:\Users\Administrator>curl "https://httpbin.org/get?city=深圳"
{
"args": {
"city": "\u6df1\u5733"
},
"headers": {
"Accept": "*/*",
"Host": "httpbin.org",
"User-Agent": "curl/8.9.1",
"X-Amzn-Trace-Id": "Root=1-69e788c8-739b25c636023a9d55ee1321"
},
"origin": "112.97.81.128",
"url": "https://httpbin.org/get?city=\u6df1\u5733"
}

It was successful, looking at the 'city' parameter value, it can be seen that 'httpbin.org' has automatically transcoded Chinese characters.

2.5 Conclusion

The 'bugpk.com' interface server receives the original UTF-8 character "city=shenzhen" for verification, and the result fails due to garbled characters and 400 Bad Requests.

The 'httpbin.org' test server also receives the UTF-8 format string "city=Shenzhen" and will actively decode it, and then return 200 successfully.

After comparative testing and conclusion deduction, it can be seen that the problem is not curl, but that different interface servers have different strictness in implementing the URL standard.

3 Solution

Use the '--data-urlencode' option to actively encode the URL before curl initiates an http get request.

curl -G "https://api.bugpk.com/api/weather" --data-urlencode "city=深圳"

4 Summary Thoughts

curl does not automatically encode Chinese characters in URLs, and garbled characters are likely to occur when using the get method and the parameters or parameter values contain Chinese characters. We cannot expect the server to provide compatibility and fault tolerance, and it is more rigorous to ensure that the data meets the standards on the local sending side.