Download Failure - Internal Server Error

asked 2017-05-15 15:35:29 -0500

updated 2017-06-10 09:19:26 -0500

Server response is slow.

Cannot complete a photo download during the day time and into the late evening. Runs aborts with the message:

Internal Server Error The server encountered an internal error or misconfiguration and was unable to complete your request. Please contact the server administrator to inform of the time the error occurred and of anything you might have done that may have caused the error. More information about this error may be available in the server error log.

Added May 19th at Noon: The server seems to be too busy to complete the task it has little problem performing in the early morning hours. I have no access to the Paragonrels server logs to determine the source of the problem. When Will this be fixed?

On May 26th the photo update run crashed twice. The server is not up to the task of effectively supporting updates except late at night. Again, when will this problem of apparently dynamically throttling server access be corrected?

On June 10th the server was totally non-responsive. The server could not even provide the data to support populating just a single MLS detail listing without throwing an "Internal Server Error".

edit retag flag offensive close merge delete

Comments

What RETS URL are you using and Account name (NO Passwords please)

bwolven gravatar imagebwolven ( 2017-05-15 15:40:57 -0500 )edit

Logon URL: http://neren.rets.paragonrels.com/ret... Login Name:dcgllc

dcgllc gravatar imagedcgllc ( 2017-05-15 16:07:58 -0500 )edit

How are you downloading images? Are you using Getobject to download binary data or URL. Or are you using the Media resource to get image URLs?

bwolven gravatar imagebwolven ( 2017-05-15 16:17:10 -0500 )edit

Using: PHRETS - PHP library for RETS, version 1.0.1 Username: del0211 Using Getobject

dcgllc gravatar imagedcgllc ( 2017-05-15 16:30:04 -0500 )edit

Are you downloading the actual binary images (Location=0)? Or are you downloading the URLs and pulling the images from that (Location=1)?

bwolven gravatar imagebwolven ( 2017-05-15 17:25:24 -0500 )edit

Downloading the actual binary images with: GetObject(Property, Photo, L_ListingID) The same queries have been running for years on the former Innovia RETS servers without issue. It is an occasional and routine problem on Paragonnels which causes a lot of rework and wasted time.

dcgllc gravatar imagedcgllc ( 2017-05-15 17:36:30 -0500 )edit

Download script was re-run at 23:00, more or less, without issue. Is there a server response issue during the daytime hours?

dcgllc gravatar imagedcgllc ( 2017-05-15 22:04:09 -0500 )edit

I'm sure that the server and network load is much heavier during normal business hours.

bwolven gravatar imagebwolven ( 2017-05-16 17:25:37 -0500 )edit

The question was not answered. The issue is that during an update that runs at night OK, fails with Internal Server Error during the daytime. The script is unchanged and is rarely run more than once per day.. Why does it crash? Are different run parameters applied to daytime and nighttime usage?

dcgllc gravatar imagedcgllc ( 2017-05-18 22:21:15 -0500 )edit

At Noon the run was nearly finished and ended with: Internal Server Error The server encountered an internal error or misconfiguration and was unable to complete your request..When will this be fixed???

dcgllc gravatar imagedcgllc ( 2017-05-19 11:05:37 -0500 )edit

What UserAgent is your application sending?

bwolven gravatar imagebwolven ( 2017-05-19 11:46:28 -0500 )edit

Also, I'm not seeing a NEREN paragon account name dcgllc? Is that still correct?

bwolven gravatar imagebwolven ( 2017-05-19 12:07:57 -0500 )edit

As stated above 2 days ago: (1) Using PHRETS - PHP library for RETS, version 1.0.1 (2) Username: del0211 (3) Using Getobject (4) login=http://neren.rets.paragonrels.com/rets/fnisrets.aspx/NEREN/login?rets-version=rets/1.7.2.

dcgllc gravatar imagedcgllc ( 2017-05-19 12:48:36 -0500 )edit

These are the last 2 getobject requests I see in our logs and they both returned 200.

2017-05-19 16:00:21 Resource=Property&Type=Photo&ID=4634756%3A%2A&Location=0 200 0 0 4293347 576 19718

2017-05-19 16:00:52 Resource=Property&Type=Photo&ID=4634608%3A%2A&Location=0 200 0 995 65536 576 8875

bwolven gravatar imagebwolven ( 2017-05-19 15:12:15 -0500 )edit

I believe you found two of the log entries for the runs I made to fix the mess the Internal Server Error made. My question is why did my 5/17 and /19 runs crash? And is something being done to avoid such in the future? I am looking for answers to these questions, not log entries.

dcgllc gravatar imagedcgllc ( 2017-05-19 23:31:08 -0500 )edit

On May 26th [today] the photo update run crashed twice. The server is not up to the task of effectively supporting updates except late at night. Again, when will this problem of apparently dynamically throttling server access be corrected?

dcgllc gravatar imagedcgllc ( 2017-05-26 15:55:32 -0500 )edit

My image downloading was taking an inordinate amount of time and discovered this discussion while investigating whether it was a server-side issue. After noticing Bruce's question on whether using Location=1 I verified my code and realized we weren't. Changing it to that, things are much faster

REW Kyle gravatar imageREW Kyle ( 2017-05-30 17:07:11 -0500 )edit

Using location=1 only provides a URL that points to a photo on the Paragon server. PHRETS skips this step and uses Getobject to provide the actual array of photos for each MLS#. The daily run cycle only downloads new photos to a local server which should not be a Paragon Server burden.

dcgllc gravatar imagedcgllc ( 2017-05-30 18:28:24 -0500 )edit

I download the images using the URL but do so in a separate process to speed up the feed update time.

REW Kyle gravatar imageREW Kyle ( 2017-05-31 12:06:10 -0500 )edit

The PHRETS run cycle is two separate runs. 1st run downloads narratives/metrics just fine -all the time. 2nd run only downloads missing photos - the run only crashes during the day/early evening evidencing overworked paragon servers.That is why the error message reports an INTERNAL SERVER ERROR.

dcgllc gravatar imagedcgllc ( 2017-06-01 15:03:52 -0500 )edit

On June 10th the server was totally non-responsive. The server could not even provide the data to support populating just a single MLS detail listing without throwing an "Internal Server Error".

dcgllc gravatar imagedcgllc ( 2017-06-10 09:19:38 -0500 )edit

Update 2017-09-07 - Are you still having issues with this?

bwolven gravatar imagebwolven ( 2017-09-07 15:04:14 -0500 )edit

Yes. Asking if the issue persists is not a proper "Answer" and should be in the "comment" thread above. As you are well aware others are posting similar issues. It is clear that Paragon does not have enough resources available during busy hours resulting in aborted sessions. What is the solution?

dcgllc gravatar imagedcgllc ( 2017-09-07 18:49:16 -0500 )edit

Are you still having the issue with internal server error? If so, how about how often is it happening? And does it happen at any particular time of day? Also what RETS transaction is it happening on: search, getobject, login, etc.?

bwolven gravatar imagebwolven ( 2017-09-19 10:35:51 -0500 )edit

The latest aborted content only run was this morning just before 10:34 EDT and before that were 5 aborted runs on 2017-09-04. I believe this is a Paragon systemic issue related to overloaded server resources likely evidenced in your server logs and empirically by my company and others in this forum.

dcgllc gravatar imagedcgllc ( 2017-09-19 10:59:56 -0500 )edit

So you are saying that you had one this morning and the last one before that was on 15 days ago?

bwolven gravatar imagebwolven ( 2017-09-19 11:10:33 -0500 )edit

Yes. What have you found in your server logs? How do you measure your server loads? Are additional resources brought on line to respond to increased traffic? What is the solution to the problem?

dcgllc gravatar imagedcgllc ( 2017-09-19 11:46:47 -0500 )edit

What is the exact error indication you received on 9/19. I checked the server logs and didn't see a single 500 error returned from your requests. They were all either 401 or 200.

bwolven gravatar imagebwolven ( 2017-09-21 12:00:40 -0500 )edit

The session timed out because the Paragon server was overloaded - likely too many users. Again, how do you measure your server loads? Are additional resources brought on line to respond to increased traffic? What is the solution to the problem?

dcgllc gravatar imagedcgllc ( 2017-09-21 12:33:48 -0500 )edit

Which session are you referring to? The HTTP session that you should be able to increase the timeout value? Or the RETS miscellaneous error saying the session timed? In which case you need to logout and back in?

bwolven gravatar imagebwolven ( 2017-09-21 13:37:33 -0500 )edit

We have dramatically increased our server timeouts to handle Paragon's overloaded server. The problem is on the Paragon end. Again & again, how do you measure your server loads? Are additional resources brought on line to respond to increased traffic? What is the solution to the problem?

dcgllc gravatar imagedcgllc ( 2017-09-22 11:18:03 -0500 )edit