spam post analysys system

Red Squirrel · Post by **Red Squirrel** » Fri Jan 07, 2005 9:09 pm

New members now go through a spam screening system and those who fall in the category of spammer are automaticly banned, and their IP is automaticly blocked.

Did this at Iceteks too, though there the system is a bit better since the post is put in the moderation queue, but for some reason I could not figure out how to make it do that here, so instead they get an error that files are missing, then when they click somewhere else, the IP block kicks in with a message saying they were classified as a spammer.

There's also some checking for trolling, but it's very basic, the most concentrated issue is spam and not trolling, but the system should also catch obvious trolls like those that post something full of swearing etc...

Don't worry, this only applies to new members. Spammers follow the same old patterns so it's really easy to detect 99.9% if they're a potential spammer or not, and probably about 95% if the post is spam or not. Though I rather have a false negative then a false positive so this certainly does not mean you won't see spam, it just means you'll see less.

Archived topic from Anythingforums, old topic ID:1620, old post ID:21590

Brooklynite · Post by **Brooklynite** » Sun Mar 06, 2005 12:37 am

Your system is a is way off base Mr. Red. I got banned and classified as a spammer, on Friday as you may recall. You then realized your mistake and corrected it. A spammer is someone who signs up just to post an ad. The closest thing I did to actual spamming was link to another forum site, but I didn't even promote it. I just mentioned it, that's all. I hope that this kind of thing does not happen again. I think you'd be better off just having your admins look for the spam rather than some mindless software.

Archived topic from Anythingforums, old topic ID:1620, old post ID:24816

Red Squirrel · Post by **Red Squirrel** » Sun Mar 06, 2005 1:04 am

Well the system is still in "test" mode the only way to really tweak it is to have in setup in a real life environment. I just keep a close eye on the Scada alarms and check what users are classified as spammers. The system is written to be very strict because of the large ammount of spammers here. Posting a link probably did it, since if I recall a link gives like 10 points. But you're safe to do it now since you're out of the "danger" zone. The filtering process is only effective after a certain point since most spammers only sign up for the sole purpose of spamming.

Actually one thing I was thinking of us trying to impliment a bayse type filter, but that would require quite allot of knowledge in AI which is not something I know allot in, so maybe something to learn later on.

Archived topic from Anythingforums, old topic ID:1620, old post ID:24819

sintekk · Post by **sintekk** » Sun Mar 06, 2005 2:14 am

Brooklynite wrote: I think you'd be better off just having your admins look for the spam rather than some mindless software.

For sure. Have the program still take logs and analyze the stuff, but don't give it the ability to act on the knowledge it acquires. That way, the program can still be tweaked until it's ready.

It's been brought up before, too.
http://www.anythingforums.com/forums/index...indpost&p=23283

Archived topic from Anythingforums, old topic ID:1620, old post ID:24822

Bookworm · Post by **Bookworm** » Sun Mar 06, 2005 10:17 am

I am curious to see how much spam we would get if you did turn that filter off. After all, if a spammer starts a thread, we can always delete it or make fun of it. I realy do the thought of banning someone because they posted something they didn't know would be classified as spam.

Archived topic from Anythingforums, old topic ID:1620, old post ID:24824

Chris Vogel · Post by **Chris Vogel** » Sun Mar 06, 2005 11:53 am

sintekk wrote: For sure. Have the program still take logs and analyze the stuff, but don't give it the ability to act on the knowledge it acquires. That way, the program can still be tweaked until it's ready.

Bonne idée! I wouldn’t mind the extra work (I would actually welcome it! My job has gotten rather uneventful!

), and I’m sure people wouldn’t mind making fun of the spam until we could get rid of it, which would only be a matter of hours at most.

Archived topic from Anythingforums, old topic ID:1620, old post ID:24825

sintekk · Post by **sintekk** » Sun Mar 06, 2005 12:44 pm

Tak wrote: (I would actually welcome it! My job has gotten rather uneventful! )

How much does a moderator position pay, anyways?

Archived topic from Anythingforums, old topic ID:1620, old post ID:24826

Red Squirrel · Post by **Red Squirrel** » Sun Mar 06, 2005 12:53 pm

The thing with this system that I like is that it teaches spammers who's boss, since they realize their post won't even get read by no one, so that's why I like the auto-ban. Later on I will reprogram it to be better and work similar to spamassassin - it will display what rules matched and will be viewable by admins. But thta's not really the highest priority so for now this basic one is staying, but I have increased the threashold which should help avoid false positives.

Archived topic from Anythingforums, old topic ID:1620, old post ID:24827

SpookyTheCat · Post by **SpookyTheCat** » Sun Mar 06, 2005 8:09 pm

sintekk wrote:
Tak wrote: (I would actually welcome it! My job has gotten rather uneventful! )

How much does a moderator position pay, anyways?

Not one dime

Archived topic from Anythingforums, old topic ID:1620, old post ID:24840

Red Squirrel · Post by **Red Squirrel** » Sun Mar 06, 2005 8:25 pm

But you get a really good pension plan.

Archived topic from Anythingforums, old topic ID:1620, old post ID:24843

Bookworm · Post by **Bookworm** » Sun Mar 06, 2005 9:06 pm

Red Squirrel wrote: But you get a really good pension plan.

Too bad retirement age is 93.

Archived topic from Anythingforums, old topic ID:1620, old post ID:24847

Brooklynite · Post by **Brooklynite** » Mon Mar 07, 2005 1:13 am

We rarely get spammers at the NYRA forums, but just recently we got one, and banned him. The admin there edited the thread. Here it is:

Instructions Inside For Being An Idiot Spammer
Damn.. I have to lay off licking toads. Makes me do all sorts of stupid things... Like coming here and spamming up this perfectly nice place.

Archived topic from Anythingforums, old topic ID:1620, old post ID:24872

Red Squirrel · Post by **Red Squirrel** » Mon Mar 07, 2005 9:01 am

Yeah I used to do that too, just edit their post. I actually pissed one off ones he kept coming back under a different name and reposting it and I'd edit it out. It was one of those stupid free ipod scams.

Archived topic from Anythingforums, old topic ID:1620, old post ID:24886

Andy · Post by **Andy** » Sat Mar 12, 2005 8:05 pm

Brooklynite wrote: Your system is a is way off base Mr. Red. I got banned and classified as a spammer, on Friday as you may recall. You then realized your mistake and corrected it. A spammer is someone who signs up just to post an ad. The closest thing I did to actual spamming was link to another forum site, but I didn't even promote it. I just mentioned it, that's all. I hope that this kind of thing does not happen again. I think you'd be better off just having your admins look for the spam rather than some mindless software.

Red i hate to say anything against you but he had a vary good point. Human interface it much better then software.

Archived topic from Anythingforums, old topic ID:1620, old post ID:25414

Cicero · Post by **Cicero** » Sun Mar 13, 2005 5:20 am

But at least with the software the spammers dont get the satisfaction of it being posted, even if there are a few mistakes.

Archived topic from Anythingforums, old topic ID:1620, old post ID:25479

Red Squirrel · Post by **Red Squirrel** » Sun Mar 13, 2005 12:33 pm

Yep and that's the idea behind it but yes it does need some tweaking. The curent one is temporarly I'll be making it better. But the last couple of tweaks might have done the trick.

Archived topic from Anythingforums, old topic ID:1620, old post ID:25497

Minnie · Post by **Minnie** » Fri Mar 25, 2005 1:46 pm

I say bring the spammers on! We can handle them!

Archived topic from Anythingforums, old topic ID:1620, old post ID:26847

sintekk · Post by **sintekk** » Wed Apr 13, 2005 5:03 pm

Couple more banned new members, I've noticed.

Archived topic from Anythingforums, old topic ID:1620, old post ID:28682

Red Squirrel · Post by **Red Squirrel** » Wed Apr 13, 2005 5:33 pm

Yeah they scored super high so it was for sure spam. Still have to get around to making this system better by making it tell me what rules match and such.

Archived topic from Anythingforums, old topic ID:1620, old post ID:28694