Ask Your Question
0

Slow photo downloads with GetObject Requests

asked 2016-07-15 11:58:13 -0500

Hi,

We have multiple FNIS/Paragon MLS feeds, and almost all of them are returning photos very slowly.

On average, we're able to download 1 photo per second. With the amount of data we're pulling across all of our MLS RETS feeds, this is eating up a lot of time. With other RETS providers, we're able to download photos at around 4-5 photos per second and sometimes faster than that.

Here's a sample of some getObject calls we're making to the MAXMLS RETs server:

[7/15/2016 6:52:58 AM] Request: ~MAXEBRDI/getobject?Type=Photo&Resource=Property&ID=40720221:3 [7/15/2016 6:53:00 AM] Request: ~MAXEBRDI/getobject?Type=Photo&Resource=Property&ID=40720221:4 [7/15/2016 6:53:01 AM] Request: ~MAXEBRDI/getobject?Type=Photo&Resource=Property&ID=40720221:5 [7/15/2016 6:53:02 AM] Request: ~MAXEBRDI/getobject?Type=Photo&Resource=Property&ID=40720221:6 [7/15/2016 6:53:04 AM] Request: ~MAXEBRDI/getobject?Type=Photo&Resource=Property&ID=40720221:7 [7/15/2016 6:53:05 AM] Request: ~MAXEBRDI/getobject?Type=Photo&Resource=Property&ID=40720221:8 [7/15/2016 6:53:06 AM] Request: ~MAXEBRDI/getobject?Type=Photo&Resource=Property&ID=40720221:9

As you can see, it's averaging about 1 photo download per second.

If you compare this to a different RETS feed of ours from a different RETS provider:

[7/15/2016 7:30:10 AM] Request: ~SFAR/getobject.aspx?Type=Photo&Resource=Property&ID=311069:2 [7/15/2016 7:30:11 AM] Request: ~SFAR/getobject.aspx?Type=Photo&Resource=Property&ID=311069:3 [7/15/2016 7:30:11 AM] Request: ~SFAR/getobject.aspx?Type=Photo&Resource=Property&ID=311069:4 [7/15/2016 7:30:11 AM] Request: ~SFAR/getobject.aspx?Type=Photo&Resource=Property&ID=311069:5 [7/15/2016 7:30:11 AM] Request: ~SFAR/getobject.aspx?Type=Photo&Resource=Property&ID=311069:6 [7/15/2016 7:30:11 AM] Request: ~SFAR/getobject.aspx?Type=Photo&Resource=Property&ID=311069:7 [7/15/2016 7:30:12 AM] Request: ~SFAR/getobject.aspx?Type=Photo&Resource=Property&ID=311069:8 [7/15/2016 7:30:12 AM] Request: ~SFAR/getobject.aspx?Type=Photo&Resource=Property&ID=311069:9 [7/15/2016 7:30:12 AM] Request: ~SFAR/getobject.aspx?Type=Photo&Resource=Property&ID=311069:10 [7/15/2016 7:30:12 AM] Request: ~SFAR/getobject.aspx?Type=Photo&Resource=Property&ID=311069:11 [7/15/2016 7:30:12 AM] Request: ~SFAR/getobject.aspx?Type=Photo&Resource=Property&ID=311069:12 [7/15/2016 7:30:12 AM] Request: ~SFAR/getobject.aspx?Type=Photo&Resource=Property&ID=311069:13

We're able to pull about 5 images per second.

Like I said above, almost all of our FNIS/Paragon RETS feeds are returning photos at about 1 photo per second. I'm wondering if there's something I'm maybe doing wrong, or if there is ... (more)

edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted
0

answered 2016-07-15 15:51:27 -0500

Pulling images one at a time using GETOBJECT can be slow due to transaction overhead and binary transfer times, especially when pulling images one at a time. The most efficient way is to pull them by URL if at all possible. Then it will better utilize the image caching capabilities. Once you have the URL, you can use it directly or download the binary files using the URL if you need to keep local copies for some reason.

Suggestions:

  • Best Solution: Get the image URLs from Paragon's Media resource. If your profile currently doesn't have access to this, contact your MLS and they should be able to enable access.

  • Second best: Pull multiple photos and listings as URLs in one transaction using GETOBJECT. For example: ID=311069:*&Location=1.

  • Pull multiple photos and listings in one transaction using GETOBJECT for example: ID=311069:,311070:. This reduces some of the overhead associated with pulling individual images.

edit flag offensive delete link more

Comments

Thanks. I will work on pulling from the Media class.

However, over the past few weeks, the our GetObject requests have gone from pulling 1 photo per second on average to 1 photo every 10-15 seconds.

I realized that you said GetObject can be slow, but how can we go from....

brokersolutions gravatar imagebrokersolutions ( 2016-08-03 07:53:23 -0500 )edit

consistently pulling 1 photo per second to taking over 10 seconds to pull 1 photo? At this rate, it will take all day to pull photo updates for 100 listings.

brokersolutions gravatar imagebrokersolutions ( 2016-08-03 07:53:36 -0500 )edit

Can you see if MAXEBRDI Getobject is performing any better yet? Also what client are you using to pull the objects?

bwolven gravatar imagebwolven ( 2016-08-05 10:48:09 -0500 )edit

It is performing better, thanks.. I am working on converting all of our FNIS/Paragon feeds to use the Media class instead of downloading the photos.

brokersolutions gravatar imagebrokersolutions ( 2016-08-05 10:52:27 -0500 )edit

I've encountered some feeds of ours where we do not have access to the Media class...is this something the MLS has to authorize?

brokersolutions gravatar imagebrokersolutions ( 2016-08-05 10:53:09 -0500 )edit

Yes. It would need to be enabled in each MLS for the profile you are using.

bwolven gravatar imagebwolven ( 2016-08-05 18:12:56 -0500 )edit
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2016-07-15 11:58:13 -0500

Seen: 941 times

Last updated: Jul 15 '16