Ask Your Question

Several markets timing out

asked 2019-08-07 12:50:22 -0600

updated 2019-08-07 13:20:41 -0600

Within the last 30min or so, several Paragon feeds are hanging on their queries and eventually time out (we have a 30min timeout set). So far I've noticed the following affected markets:

Is there a known issue? It hasn't been very long and not sure if it'll go away on its own shortly, but just wanted to post since we noticed. Thanks!

Edit: More markets are affected now:

edit retag flag offensive close merge delete


Usernames if needed: 206539027, NTPV16653, MIRblueroof, 6268

MattMessmer gravatar imageMattMessmer ( 2019-08-07 12:52:17 -0600 )edit

1 Answer

Sort by ยป oldest newest most voted

answered 2019-08-07 14:06:20 -0600

updated 2019-08-07 14:30:38 -0600

Not getting reports of other vendors having issues.
Are you still getting timeouts?
Do you know what specific transaction is timing out?
Is it login, searches, etc?
Have you verified that it isn't an issue with your ISP?

edit flag offensive delete link more


Yep, queries are still hanging. I most commonly see open house queries timing out. At that point in our code, we should already be logged in. Shouldn't be an ISP issue because we're able to download updates from other non-Paragon markets.

MattMessmer gravatar imageMattMessmer ( 2019-08-07 14:58:03 -0600 )edit

Digging into this more, I'm unable to reproduce the issue locally, so maybe it is on my end. Still investigating and will post again when I'm done.

MattMessmer gravatar imageMattMessmer ( 2019-08-07 15:03:30 -0600 )edit

This is weird. I can't repro locally but it's still only happening with Paragon feeds, most of them get stuck on open house downloads. Is there any rate limiting in place? Wondering why it would only affect the production servers.

MattMessmer gravatar imageMattMessmer ( 2019-08-07 15:32:19 -0600 )edit

Don't know. When you run locally are you using the same ISP?

bwolven gravatar imagebwolven ( 2019-08-07 15:45:47 -0600 )edit

Nope, our production servers are hosted in Microsoft Azure.

MattMessmer gravatar imageMattMessmer ( 2019-08-07 15:54:16 -0600 )edit

I just saw a timeout on login for NEREN, if that helps. Seems intermittent though.

MattMessmer gravatar imageMattMessmer ( 2019-08-07 16:04:58 -0600 )edit

Looking at MAXEBRDI in our logs. In the last 24 hours I've only seen 2 transactions over 30 seconds.
One was a little over 30 seconds the other is a little over 60 and they were both property.

bwolven gravatar imagebwolven ( 2019-08-07 16:13:47 -0600 )edit

My apologies, looking further, I think these are login failures. It's hard to tell in libRETS.

MattMessmer gravatar imageMattMessmer ( 2019-08-07 16:23:01 -0600 )edit

I've confirmed these are login failures. I was able to alleviate the symptoms by setting a separate timeout for logins, only 30 seconds. This helps our servers not get stuck on these at least, but this is still an issue.

MattMessmer gravatar imageMattMessmer ( 2019-08-07 18:16:50 -0600 )edit

So all of the ones you listed above are having intermittent login errors?

bwolven gravatar imagebwolven ( 2019-08-07 18:18:43 -0600 )edit

Yep, and more markets beyond that, too. But yeah it is intermittent, I just saw a failure for a calrets.mlslistings at 4:04pm PDT, but it just tried again at 4:18 and it worked. Thanks for looking into this, I know intermittent issues are painful to troubleshoot.

MattMessmer gravatar imageMattMessmer ( 2019-08-07 18:26:11 -0600 )edit

Are you using the same IP or group of IP addresses for all of them?
Wondering if it might be an issue with a particular IP address or something.
I'll have to look more tomorrow.

bwolven gravatar imagebwolven ( 2019-08-07 18:34:38 -0600 )edit

Yeah, all the production servers are behind the same load balancer (same IP). Locally I'm using a different IP. No worries, it can wait. The timeout change I made mitigated most of the symptoms, we're not nearly as far behind now. Thanks again!

MattMessmer gravatar imageMattMessmer ( 2019-08-07 19:29:09 -0600 )edit

This is still happening for several markets but it's still intermittent. Simply retrying seems to fix the issue. As a result, it's not really a problem for us, but just wanted to let you know because I also didn't want to needlessly spam login requests :)

MattMessmer gravatar imageMattMessmer ( 2019-08-08 12:51:32 -0600 )edit

Thanks for letting us know.
Make sure you put some kind of delay before retries.
Also limit the number of retries so it doesn't get stuck in retry loops hammering the servers.

bwolven gravatar imagebwolven ( 2019-08-08 14:45:28 -0600 )edit

Can you verify that the URLs you are using are formatted correctly.
Especially the ones you use after login.

bwolven gravatar imagebwolven ( 2019-08-15 16:12:58 -0600 )edit
Login/Signup to Answer

Question Tools

1 follower


Asked: 2019-08-07 12:50:22 -0600

Seen: 581 times

Last updated: Aug 07 '19