CDN for image urls giving 504 error
Images are failing to load on our feed processing server. CloudFront, the CDN Paragon is using, is saying that it can't establish a connection. We can receive the images when using a computer with a different IP address.
The request Could not be satisfied.
Cloudfront attempted to establish a connection with origin, but either the attempt failed or the origin closed the connection. We can't connect to the server for this app or website at this time ...
What RETS URL and account name are you using (NO PASSWORDS).
Can you provide one of the failing photo URLs?
We dug more into the issue. It's an issue when the image domain resolves to IPs in the 184.108.40.206 subnet.
For the computer that we use to process images, cdnparap50.paragonrels.com resolves to 220.127.116.11. On other computers it resolves to 18.104.22.168.
Not sure why you are getting that error.
The URL you included displays the image without issue.
The "cdnparap50.paragonrels.com" resolves to 22.214.171.124 for me.
Since it's CDN it should resolve to different servers based on location.
So your image download server is in a different location than the other computers?
I'm in Arizona, my office is in California. Our image server is in a colocation separate from the office.
A quick google of "CloudFront 504" shows that they often come up from a config or firewall issue on your end. https://docs.aws.amazon.com/AmazonClo....
Are there any rules on image pulling that blocks IP addresses on repeated requests to access the same resource?
The only ones I know of are listed here but I thought it returned a 403 error:
I have the same 504 error when trying to download images (CentOS server)... It times out... We installed XAMMP on a local PC and connected to our CentOS grabber server's DB and using the same exact script we are able to download. We ran the script on another CentOS server we have and it also got the 504 error...
Any blocking that is done would be by IP address.
Is your second CentOS box resolving the CDN ip to the same 13.* address?
Here is an older screen cap from last week...I will get a screen cap from server 2 for you today... Any other things you would like me to test and share? https://ibb.co/6vGK52z
Have you tried one of the failing URLs in a browser on the CentOS boxes?
I don't have any GUI setup on those machines... terminals only... :)
I have been grabbing data for 5 years.. never had this problem... until last week...
I don't know if it would be helpful, but if you give me a list of the domains you'd like me to query to compare I can do that... The 13.35 network is the wrong dns resolving?
"I have been grabbing data for 5 years.. never had this problem... until last week..." Same here. Bruce, I think you're going to see this issue popping up much more frequently. It seems like timeout is between CloudFront and Paragons image server. Maybe some CloudFront CDN servers can't access the Paragon server? Whats odd too is that I was able to get these images on my local machine yesterday but now I'm seeing the 504 error.
Bruce, this might be useful for reproducing the issue.
If I take this example image: https://cdnparap50.paragonrels.com/Pa...
Then I nslookup the domain on it:
$ nslookup cdnparap50.paragonrels.com
Server: 126.96.36.199 Address: 188.8.131.52#53
Non-authoritative answer: cdnparap50.paragonrels.com canonical name = d2azcccpalp73y.cloudfront.net. Name: d2azcccpalp73y.cloudfront.net Address: 184.108.40.206 ...
You can see the IP address I get for that domain. If you use curl to get image and have the domain resolve to the same IP address I get for the domain, then you can get the same timeout issue.
The curl command outputs:
The request could not be satisfied.
CloudFront attempted to establish a connection with the origin, but either the attempt failed or the origin closed the connection. We can't connect to the server for this app or website at this time. There might be too much traffic or a configuration error. Try again later, or contact the app or website owner. Generated by cloudfront (CloudFront)
I tried your image same result: # wget https://cdnparap50.paragonrels.com/Pa... --2021-12-21 18:38:56-- https://cdnparap50.paragonrels.com/Pa... Resolving cdnparap50.paragonrels.com (cdnparap50.paragonrels.com)... 220.127.116.11, 18.104.22.168, 22.214.171.124, ... Connecting to cdnparap50.paragonrels.com (cdnparap50.paragonrels.com)|126.96.36.199|:443... connected. HTTP request sent, awaiting response... 504 Gateway Time-out 2021-12-21 18:39:26 ERROR 504: Gateway Time-out.
2nd server same result: # wget https://cdnparap50.paragonrels.com/Pa... --2021-12-21 18:42:28-- https://cdnparap50.paragonrels.com/Pa... Resolving cdnparap50.paragonrels.com (cdnparap50.paragonrels.com)... 188.8.131.52, 184.108.40.206, 220.127.116.11, ... Connecting to cdnparap50.paragonrels.com (cdnparap50.paragonrels.com)|18.104.22.168|:443... connec
Can you try the same image using HTTP instead of HTTPS?
I heard from my MLS that Paragon made some mods on their end.. I tested a few images on my 2 servers and they are downloading... Paragon needs an official support channel... This forum is really unacceptable for official support... Please pass on my thanks for addressing the issue.. Some details would be nice but probably beyond the scope of this organization... Merry Christmas!
Thanks for the update. I am able to get photos now as well.
Thanks for the response.