|
OriginalGriff wrote: mood changes, delirium, even what looks like Alzheimer's!
That would really p#*s me off. Hold on a second...
|
|
|
|
|
Was missing you in the daily Wordle and was wondering how you are doing.
Good to see your note.
|
|
|
|
|
|
Several months back, we got a new customer using a cloud-based POS system that we hadn't worked with before. They are actually the first vendor we have encountered that had a real API for which they gave us a key and basically just said 'have fun!'.
I'm used to figuring things out myself so no problem, after all, it's just a web request that gets a response and does something with it. I dug in and using their Swagger UI found 90% of what I was looking for and started pulling data by the month. There are 2 different requests/pulls, 1 returns around 120K records/month and the other around 240K records/month. These ran fine until last weekend when they both started timing out.
I decided to break up the requests into smaller parts to maybe help things along so I added an outside loop to request one date at a time. Nope, still timing out. Increase the read timeout in the header. Still times out so double it...still times out so double it...
Now I'm up to a read timeout of 3,000,000 (50 minutes!!!) and it's finally running, albeit very slowly and still with some timeouts that I am currently manually re-running. It won't do any good to increase that value anymore as the timeouts are coming @ 2 minutes likely due to a server setting beyond my control.
It kind of worked up to 38K recs, then flat out refused to go any further. I just wrapped the request in a conditional loop so it can't get out until it actually finishes, or I manually stop it.
7 hours later and it's been hitting their server all night and no more progress...oh well, just keep on trying then. I wonder if they every check their logs?
"Go forth into the source" - Neal Morse
"Hope is contagious"
|
|
|
|
|
I wonder when you'll get the DOS attack attempt notification. 
/edit: forgot to mention : I feel your pain !
modified 12-Sep-22 11:32am.
|
|
|
|
|
whats the back end db? ............
Caveat Emptor.
"Progress doesn't come from early risers – progress is made by lazy men looking for easier ways to do things." Lazarus Long
|
|
|
|
|
|
abmv wrote: whats the back end db? ............
I have absolutely no idea. The tables are exposed as objects in the Swagger UI, requests are made using OData (v.1 apparently as aggregate extensions are not supported) and responses come back as JSON.
The query/request I'm trying to run is not complicated. (7 columns from 2 tables with joined on a PK/FK field.)
Maybe they need to wind up the rubber bands? 
"Go forth into the source" - Neal Morse
"Hope is contagious"
|
|
|
|
|
I guess it begs the question: what does a client do with 120-240K records that a query on the server can't handle? And how many fields and bytes are we talking about?
"Before entering on an understanding, I have meditated for a long time, and have foreseen what might happen. It is not genius which reveals to me suddenly, secretly, what I have to say or to do in a circumstance unexpected by other people; it is reflection, it is meditation." - Napoleon I
|
|
|
|
|
Gerry Schmitz wrote: what does a client do with 120-240K records that a query on the server can't handle?
I didn't really elaborate on the fact that the records come in chunks of 2K. I've tried aggregates but the server doesn't recognize the $apply directive so I'm guessing they are using an old version of OData on the server.
Gerry Schmitz wrote: how many fields and bytes are we talking about?
7 fields from 2 joined tables (highly normalized db btw)
"Go forth into the source" - Neal Morse
"Hope is contagious"
|
|
|
|
|
Maybe it's the wrong door. I would try FTP'ing the whole thing.
"Before entering on an understanding, I have meditated for a long time, and have foreseen what might happen. It is not genius which reveals to me suddenly, secretly, what I have to say or to do in a circumstance unexpected by other people; it is reflection, it is meditation." - Napoleon I
|
|
|
|
|
A man goes to the dentist to ask how much it would be to pull a tooth. “$100,” said the dentist. “Oh, that’s expensive,” said the main. “Do you have anything cheaper?” “That’s the normal price for an extraction,” said the dentist. The man thinks about it, “what about if you don’t use the anesthetic?” “Well, that would be unusual, but we could do that. It would be about $75.” The man thinks some more. “What about if you used a trainee and no anesthetic?” “Well,” said the dentist, ”I think that could work, but it would be a lot more painful. I think that would be about $35.” The man thought some more. “That’s still a lot. What if you make it a training session with a student doing the extraction, and the other students can watch?” The dentist says, “Ok, that would be good for the students, but it will be traumatic to have it done that way. I’ll charge you $5 for that.” “Great,” said the man. “That’s perfect. Can I book my wife for her appointment on Wednesday?
PartsBin an Electronics Part Organizer is finally available for download.
JaxCoder.com
|
|
|
|
|
Have you discovered a very old box of Christmas crackers ?
Life should not be a journey to the grave with the intention of arriving safely in a pretty and well-preserved body, but rather to skid in broadside in a cloud of smoke, thoroughly used up, totally worn out, and loudly proclaiming “Wow! What a Ride!" - Hunter S Thompson - RIP
|
|
|
|
|
No but I have found how uncomfortable the dog house is!
PartsBin an Electronics Part Organizer is finally available for download.
JaxCoder.com
|
|
|
|
|
pkfox wrote: a very old box of Christmas crackers
More like damp squibs...
Freedom is the freedom to say that two plus two make four. If that is granted, all else follows.
-- 6079 Smith W.
|
|
|
|
|
The equivalent in my day was barber school (not my choice) ... they could still nick you with an electric razor.
"Before entering on an understanding, I have meditated for a long time, and have foreseen what might happen. It is not genius which reveals to me suddenly, secretly, what I have to say or to do in a circumstance unexpected by other people; it is reflection, it is meditation." - Napoleon I
|
|
|
|
|
Sombre service can be amusing time to laugh (7)
Life should not be a journey to the grave with the intention of arriving safely in a pretty and well-preserved body, but rather to skid in broadside in a cloud of smoke, thoroughly used up, totally worn out, and loudly proclaiming “Wow! What a Ride!" - Hunter S Thompson - RIP
|
|
|
|
|
Funeral?
Sombre service = definition
can be = Double Definition
amusing = Fun
time to laugh = eral (The age of)
|
|
|
|
|
Yay YAUT
Life should not be a journey to the grave with the intention of arriving safely in a pretty and well-preserved body, but rather to skid in broadside in a cloud of smoke, thoroughly used up, totally worn out, and loudly proclaiming “Wow! What a Ride!" - Hunter S Thompson - RIP
|
|
|
|
|
I find myself in need of enlightenment. When I tried to solve this I guessed FUNERAL basing it solely off a def of "sombre service" and "amusing" for FUN. But I could find no reference to ERAL that meant "time to laugh" or "the age of" anything. Could you perhaps point me in the right direction? Thank you.
|
|
|
|
|
I'm the Solver, you should be asking the Setter. 
It's possible that my puzzle solution this morning was incomplete. This looks more correct:
Sombre service = definition
can be = Double Definition
amusing = Fun
time = era
to = junction indication
laugh = L
L - Laugh in Internet Slang
If you ever want to design a puzzle that I won't be able to solve, just use internet slang.
You will have to ask Pete if it's correct.
KMAGYOYO,
-David Delaune
|
|
|
|
|
No, no need. I'm sure you're correct. As soon as I saw the L pulled off leaving ERA I face-palmed so hard there are fingerprints on my forehead.  So obvious. Thank you.
|
|
|
|
|
L is just the first letter of laugh I don't do internet slang either.
Life should not be a journey to the grave with the intention of arriving safely in a pretty and well-preserved body, but rather to skid in broadside in a cloud of smoke, thoroughly used up, totally worn out, and loudly proclaiming “Wow! What a Ride!" - Hunter S Thompson - RIP
|
|
|
|
|
And it is the price of hamburgers.
We had two hamburgers with fries and a drink in the town of Blavand and found out afterwards that our bank account was 60 Euros lighter! 
(the hamburgers were of good quality btw)
|
|
|
|
|
Be fair - the drink was a bottle of wine each ...
"I have no idea what I did, but I'm taking full credit for it." - ThisOldTony
"Common sense is so rare these days, it should be classified as a super power" - Random T-shirt
AntiTwitter: @DalekDave is now a follower!
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
