Coders and Scripters: Need your opinion.

katamaster81899

Alright, so I've been poking around at the idea of creating some sort of dev tracker, possible a twitter account that tweets the username, handle, and title of the post like the former neverwinter one https://twitter.com/NW_DevTracker.

I was thinking, is it possible to create a script that periodically (once an hour, or something of the sort) checks a list of users that we specify (aka devs, such as @strumslinger @pwlaughingtrendy , etc), and tweets if they see that there is a new comment posted. If there is, it will tweet the information I specified above.

I'm still a noobie coder (only 16 years old, been coding for 4 years, know Java, JS, HTML5, Python, and VB.net), but I'm not really sure how I'd go about creating something like this.

Can anyone more experienced tell me how viable it is, and where I'd go about starting something like this?

primar13

The updated Dev Tracker has already been suggested, and I believe is in development.

Might want to check with @Askray... Think he was leading the charge. but who knows

nrglg

katamaster81899 wrote: »

Can anyone more experienced tell me how viable it is...

To answer the question, yes it is possible. Is it easy? Fairly. In a handful of years if you continue you'll have no problem making something like that. I started programming when I was 9 and am now a lot older. I'd say the 5-year mark was when my skill started really picking up. By year 10 you will be awesome. By year 15, if you don't dream code in your sleep yet, you might want to quit programming. That's a necessary skill for the best programmers. Usually picked up within the first 3-years (I'm not even joking).

katamaster81899 wrote: »

...and where I'd go about starting something like this?

Step 1. Use Google like crazy. It's the best tool for learning nowadays in combination with StackOverflow.
Step 2. Learn how to communicate over protocols, learn XML, how to parse (in general and including how to parse XML), regex, and how to communicate with REST APIs. Just to name a few things you'll need.
Step 3. ???
Step 4. Profit

Good luck on your journey. You probably aren't ready to do something like this if you had no idea where to start.

frtoaster

Your options from easiest to hardest are as follows:

1. Have PWE enable the Vanilla API and then use that. From Vanilla's documentation though, it seems that PWE has to enable advanced search as well if you want to query by author.

http://blog.vanillaforums.com/api-search/

Correction: If they enable the API, you wouldn't need advanced search.

http://blog.vanillaforums.com/api/api-users-comments/

Also, as I said in another thread, I have some reservations about enabling the API.

frtoaster wrote: »

If they do decide to enable the API, I seriously hope they test its security before releasing it to the public. We've already seen leaked IP addresses and private messages, so it's clear that they didn't rigorously test the forums after the migration.

I'm not saying that they shouldn't enable it, just that they should be careful.

2. Have PWE enable advanced search and query the search page directly, like so:

http://vanillaforums.org/search?author=Linc,Todd

3. Directly query the search page as it is now:

http://perfectworld.vanillaforums.com/search?Search=katamaster81899

Note that this will return not only posts written by a user, but also any posts that contain that user's name, so you'll have to manually filter out posts by other people.

4. Use a web crawler to crawl the forums and look for posts by specific users. You can write your own crawler or use an existing one that allows hooks for custom code.

Solutions 1 to 3 depend on how often Vanilla updates the index of its search engine. With solution 4, you control how often you crawl the forums. But no matter which solution you choose, Vanilla Forums may decide to block you if they find that you are making too many requests.

Your best bet right now is probably solution 3. Most languages have libraries that allow you to make HTTP requests and parse the responses. Of course, you might want to wait to see if PWE writes their own dev tracker or enables advanced search.

nrglg

PWE's old forums had virtually no limit on requests. My crawler ate through a huge portion of their forum in a single day.

frtoaster

nrglg wrote: »

PWE's old forums had virtually no limit on requests. My crawler ate through a huge portion of their forum in a single day.

Vanilla Forums might do things differently though. Many websites allow unlimited crawling from the big search engines like Google, Yahoo, and Microsoft, but see other robots as nuisances, so they block them if they see too many requests.

frtoaster

OK, I really should read more before posting. The API is actually up and running.

http://perfectworld.vanillaforums.com/api/v1/users/comments?User.Name=pwlaughingtrendy
http://perfectworld.vanillaforums.com/api/v1/users/comments.xml?User.Name=pwlaughingtrendy
http://perfectworld.vanillaforums.com/api/v1/users/comments.json?User.Name=pwlaughingtrendy

I'm not so sure this is a good thing. They've disabled access to /users/sso, but a spammer or hacker who's less discriminating could still get information that you wouldn't necessarily want him to have.

http://perfectworld.vanillaforums.com/api/v1/users/sso?User.Name=pwlaughingtrendy

I've sent a PM to pwlaughingtrendy and trailturtle about this.