Don't make me register for an "API key" or whatever you call it just so you can get my personal info and log, throttle, or charge for my requests for a subset of the data that I can already get freely via your app or website. Don't spread the misconception that your "developer API" is the only way to get at your data.
To "developers":
Don't beg and whine for an API when you can just use your fucking browser or their app to see the data already. Learn what View Source can do for you. Don't be mislead by web service providers who peddle their stupid API bullshit just so they can exert more control over you.
Name:
Anonymous2015-07-13 10:38
Enjoy data bloat when you parse 400% more data than necessary.
This thread sucks harder than Cudder in the port-a-john pulled up to the summer Intel employee barbecue
just so you can get my personal info
Because lists of the email addresses of autists are in high demand
log, throttle
you are retarded if you believe that sane people won't try to cover their ass in these extremely common and simple ways
charge
you are similarly/further retarded if you believe that companies/organizations providing/developing an API will not try to squeeze every cent out of its use
for a subset of the data that I can already get freely via your app or website
Of course! everyone should just write html scrapers--using what's already there, no matter how many ebin hacks are necessary, is always better!
don't beg and whine for an API when you can just use your fucking browser Learn what View Source
Holy shit IHBT. Where's your HTML parser? You got one in ASM? I have some markup for it to look at whenever you feel like publishing what I'm sure is a monstrous, broken aberration against all that is good. Since you won't ever do that though, in the meantime, you can take this "every integration is a hostile integration" attitude and fucking choke to death on it. RRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRREEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEe
Name:
Anonymous2015-07-13 18:03
I parse data from HTML. Further, I use regex to do it. People at my office give me odd looks for doing it, and when I ask them why, they just tell me something about satan or whatever. I think they got that meme from stack overflow.
most regex engines are more than simple regular expression parsers
Maybe the cancerous ones
Your point being?
I'll spell it out for you, pinhead: Square peg, round hole
>>9 You should realize by now that absolutely nothing is one off. Have fun dicking with your RE every time there is a subtle markup change or inconsistency.
Regex engines that implement lookahead/lookbehind are able to parse more than just regular languages. Pretty much every decent engine out there supports them.
I'll spell it out for you, pinhead: Square peg with rounded corners, square hole.
Cancer! Why even use RE at that point? Might as well write a whole goddamned lexer, at least that has some chance of being legible. Christ I hate REfags.
>>19 I don't think we were ever above it, but enough of us didn't take it seriously at least. We have a new vocal 4channer so he could explain the new tone.
>>10,12,13 Propose an alternative that's better then. Keep in mind the alternative needs to have the same performance. Only the data you want is extracted. Everything else is passed over. The criteria are performance and ease of use.
>>35 So your solution to the changing html problem is to hope that the web developer doesn't change the structure? That's a fantastic and stable plan yo!
>>32,39 Those changes are easy to handle. Once you've written a scraper that extracts what you want, updating it for a new site structure is just a matter of changing a few paths and expressions. Scraping becomes difficult when the site appers make things deliberately hard for your bot. Like randomizing the html structure of the page or captchas. The other time when its hard is when the appers can't produce valid html, or the data isn't organized in any reliable way.
I pictured >>39 as a child in a plastic high set at a table in a fast food restaurant throwing opened packets of mustard and ketchup at me as he said that.
Name:
Anonymous2015-07-15 9:00
Smoke weed everyday
Name:
Anonymous2015-07-15 18:19
>>30 The API is rarely documented and usually hidden in megabytes of Javashit cruft. I once tried to reverse https://volafile.io's API, it was awful.
Name:
Anonymous2015-07-15 18:21
>>43 Just stay determined. Reverse engineering is tedious but it has a reward to it. Like checking these sweet dubs.
Name:
Cudder !cXCudderUE2015-07-16 1:15
>>43 Protip: look at the requests being made, it's usually obvious from that.
so you can get my personal info and log, throttle, or charge for my requests
and you think they aren't doing that when you GET their webpages? lol. if they give you an API, you can still hide behind a VPN and give them a fake name/pay with a prepaid debit card.
Learn what View Source can do for you.
most websites are migrating to being fully Javascript rendered e.g., React. to use that you'll need browser automation. so in addition to whatever parsing code you need, you'll have to pull down most of WebKit (or whatever) just to extract a little bit of data on a web page. G-d help you if they randomly change one of the CSS selectors.
it's just an extremely brittle design. most people need on-demand, reliable data and don't mind paying for access because they're not mentally ill trannies in some Eastern European hovel.