Slow photo downloads with GetObject Requests

retag add tags

Hi,

We have multiple FNIS/Paragon MLS feeds, and almost all of them are returning photos very slowly.

On average, we're able to download 1 photo per second. With the amount of data we're pulling across all of our MLS RETS feeds, this is eating up a lot of time. With other RETS providers, we're able to download photos at around 4-5 photos per second and sometimes faster than that.

Here's a sample of some getObject calls we're making to the MAXMLS RETs server:

[7/15/2016 6:52:58 AM] Request: ~MAXEBRDI/getobject?Type=Photo&Resource=Property&ID=40720221:3 [7/15/2016 6:53:00 AM] Request: ~MAXEBRDI/getobject?Type=Photo&Resource=Property&ID=40720221:4 [7/15/2016 6:53:01 AM] Request: ~MAXEBRDI/getobject?Type=Photo&Resource=Property&ID=40720221:5 [7/15/2016 6:53:02 AM] Request: ~MAXEBRDI/getobject?Type=Photo&Resource=Property&ID=40720221:6 [7/15/2016 6:53:04 AM] Request: ~MAXEBRDI/getobject?Type=Photo&Resource=Property&ID=40720221:7 [7/15/2016 6:53:05 AM] Request: ~MAXEBRDI/getobject?Type=Photo&Resource=Property&ID=40720221:8 [7/15/2016 6:53:06 AM] Request: ~MAXEBRDI/getobject?Type=Photo&Resource=Property&ID=40720221:9

As you can see, it's averaging about 1 photo download per second.

If you compare this to a different RETS feed of ours from a different RETS provider:

[7/15/2016 7:30:10 AM] Request: ~SFAR/getobject.aspx?Type=Photo&Resource=Property&ID=311069:2 [7/15/2016 7:30:11 AM] Request: ~SFAR/getobject.aspx?Type=Photo&Resource=Property&ID=311069:3 [7/15/2016 7:30:11 AM] Request: ~SFAR/getobject.aspx?Type=Photo&Resource=Property&ID=311069:4 [7/15/2016 7:30:11 AM] Request: ~SFAR/getobject.aspx?Type=Photo&Resource=Property&ID=311069:5 [7/15/2016 7:30:11 AM] Request: ~SFAR/getobject.aspx?Type=Photo&Resource=Property&ID=311069:6 [7/15/2016 7:30:11 AM] Request: ~SFAR/getobject.aspx?Type=Photo&Resource=Property&ID=311069:7 [7/15/2016 7:30:12 AM] Request: ~SFAR/getobject.aspx?Type=Photo&Resource=Property&ID=311069:8 [7/15/2016 7:30:12 AM] Request: ~SFAR/getobject.aspx?Type=Photo&Resource=Property&ID=311069:9 [7/15/2016 7:30:12 AM] Request: ~SFAR/getobject.aspx?Type=Photo&Resource=Property&ID=311069:10 [7/15/2016 7:30:12 AM] Request: ~SFAR/getobject.aspx?Type=Photo&Resource=Property&ID=311069:11 [7/15/2016 7:30:12 AM] Request: ~SFAR/getobject.aspx?Type=Photo&Resource=Property&ID=311069:12 [7/15/2016 7:30:12 AM] Request: ~SFAR/getobject.aspx?Type=Photo&Resource=Property&ID=311069:13

We're able to pull about 5 images per second.

Like I said above, almost all of our FNIS/Paragon RETS feeds are returning photos at about 1 photo per second. I'm wondering if there's something I'm maybe doing wrong, or if there is ... (more)

brokersolutions

asked 2016-07-15 11:58:13 -0500

edit flag offensive 0 remove flag close merge delete

Comments

add a comment see more comments

1 Answer

Sort by

oldest

newest

most voted

Pulling images one at a time using GETOBJECT can be slow due to transaction overhead and binary transfer times, especially when pulling images one at a time. The most efficient way is to pull them by URL if at all possible. Then it will better utilize the image caching capabilities. Once you have the URL, you can use it directly or download the binary files using the URL if you need to keep local copies for some reason.

Suggestions:

Best Solution: Get the image URLs from Paragon's Media resource. If your profile currently doesn't have access to this, contact your MLS and they should be able to enable access.
Second best: Pull multiple photos and listings as URLs in one transaction using GETOBJECT. For example: ID=311069:*&Location=1.
Pull multiple photos and listings in one transaction using GETOBJECT for example: ID=311069:,311070:. This reduces some of the overhead associated with pulling individual images.

bwolven

answered 2016-07-15 15:51:27 -0500

edit flag offensive 0 remove flag delete link

Comments

Thanks. I will work on pulling from the Media class.

However, over the past few weeks, the our GetObject requests have gone from pulling 1 photo per second on average to 1 photo every 10-15 seconds.

I realized that you said GetObject can be slow, but how can we go from....

brokersolutions (2016-08-03 07:53:23 -0500) edit

consistently pulling 1 photo per second to taking over 10 seconds to pull 1 photo? At this rate, it will take all day to pull photo updates for 100 listings.

brokersolutions (2016-08-03 07:53:36 -0500) edit

Can you see if MAXEBRDI Getobject is performing any better yet? Also what client are you using to pull the objects?

bwolven (2016-08-05 10:48:09 -0500) edit

It is performing better, thanks.. I am working on converting all of our FNIS/Paragon feeds to use the Media class instead of downloading the photos.

brokersolutions (2016-08-05 10:52:27 -0500) edit

I've encountered some feeds of ours where we do not have access to the Media class...is this something the MLS has to authorize?

brokersolutions (2016-08-05 10:53:09 -0500) edit

Yes. It would need to be enabled in each MLS for the profile you are using.

bwolven (2016-08-05 18:12:56 -0500) edit

add a comment see more comments

Slow photo downloads with GetObject Requests edit

Comments

1 Answer

Comments