VA CAAR - Missing some photo updates

retag add tags

Account name: BLUEROOF360

I'm pretty sure this is on my end, I just need help trying to figure out why. We occasionally miss photos for new listings and I can't figure out why. The only pattern I've noticed is that in most cases, the photos seem to show up before the listing does (i.e. L_InputDate > L_Last_Photo_Updt), but I don't think that matters.

Using VA CAAR as an example (but I doubt this problem is specific to this feed). We query for photo updates using the Media resource. A query for the last hour of updates looks like this:

(L_Last_Photo_updt=2019-03-08T17:50:19+00:00-2019-03-08T18:50:24+00:00)

We use this to identify which listings changed, then we do another query to get all of the URLs for each listing affected (replacing all URLs for a listing like this ensures deleted photos get removed too).

Our incremental downloads pick the next min date by saving the max update date of the results of this download.

Does anything about this sound wrong? Do we need to do any "overlapping" in our date range downloads (like if there's any sort of replication lag, for example)? I'm at a loss...

Thanks much for your time!

Edit: In case this is useful, here are a couple displayIds we missed photos for, but got the listing just fine:

586742

586853

Note that this doesn't affect all new listings, just some.

MattMessmer

asked 2019-03-08 13:13:20 -0500, updated 2019-03-08 13:14:59 -0500

edit flag offensive 0 remove flag close merge delete

Comments

add a comment see more comments

1 Answer

Sort by

oldest

newest

most voted

So, I reviewed a listing, and wanted to provide sort of a time line of photo uploads to it, hopefully it will help show the lifecycle of listing photos: For 586742, our photo audits show the following: there were 8 photos uploaded at 2019-03-06 04:45:00, about 13 added 2019-03-06 04:46:00,~15 added 2019-03-06 04:47:00,3 at 2019-03-06 04:48:00 and two at 2019-03-06 05:04:00.

At each of these stages, the L_Last_Photo_updt was updated to that time period, guessing the agent likely uploaded the photos in batches. But the photo update date does truncate the milliseconds from the time stamp. So for example on 586742, the first record of the Last Photo Update Date was 2019-03-06 04:45:01.7970344. So my thought is maybe roll back your times to account for the possible difference in milliseconds.

Let me know if you have questions!

Thank you,

Sadie

answered 2019-03-11 17:26:11 -0500

edit flag offensive 0 remove flag delete link

Comments

Thanks Sadie, I'll see if I can figure out if the milliseconds are throwing us off!

MattMessmer (2019-03-11 18:27:09 -0500) edit

I added an overlapping fetch of 5min to make sure any rounding/truncation wasn't an issue. Unfortunately we missed photos within the last day. L_DisplayId 587945. One discrepancy I'm wondering about is that the agent said they added the listing at 9:46pm (I assume Eastern time), but this conflicts with the timestamps on the record. Very confusing.

MattMessmer (2019-03-28 13:33:56 -0500) edit

I also noticed this listing's photos were uploaded before the listing itself was. Wondering if that has anything to do with it, too.

MattMessmer (2019-03-28 13:34:45 -0500) edit

One other thought. Is there any issue to downloading updates via the Media resource instead of the Property resource? I was thinking about trying the Property resource to see if that would help. But, it's just a guess.

MattMessmer (2019-03-28 13:49:05 -0500) edit

Timestamps returned by Paragon RETS are in UTC.
The Media resource should work fine.
But if you do try Property with GETOBJECT, please use Location=1 to get URLs.
This will help to avoid increasing system load.

bwolven (2019-03-28 14:23:39 -0500) edit

add a comment see more comments

VA CAAR - Missing some photo updates edit

Comments

1 Answer

Comments