Sciencemadness Discussion Board - Tired of reporting spam

What advantages can the spammers get when they include a string of random letters in the title?

It is a numbers game. They employ a wide range of simple strategies in the hopes that something will evade detection for long enough for it to have its desired effect. It is the same reason that ferns produce millions of spores and fish lay thousands of eggs. All they need is for one to stick. It is more time effective to produce many simple variants and very large volumes than it is to do any actual targeted marketing or evaluation of strategies.

12thealchemist - 20-12-2018 at 12:22

I've noticed a significant increase in the quantity of spam slipping through the net. It seems the majority is getting tacked on the end of threads started within the past decade or so, and was wondering why Melgar's excellent filter is missing these. Are we looking at a move soon?

fusso - 20-12-2018 at 12:30

@Melgar had you analyse % of spam posts attached to the end of real threads and spam posts as new threads? Are they different now (recent flood of slipped spams at the end of real threads) and then (before the flood)?

j_sum1 - 30-12-2018 at 16:15

Right. Back from holiday. Seems to be a regular stream of spam making its way through at the moment and it is taking a bit to clean.

Current trends:

Spam appended to the end of sticky threads: often ten or more at a time.
Spammers evading deletion through use of registrations from months ago.
Spam containing chemistry-related words along with all the other garbage. Some words and phrases seem to be harvested from this board.
General decrease in efficiency of botkilla in my estmation: I think this dates from when Melgar loosened the filter following an accidental thread deletion. It might be time to tweak those flag weightings.

At the moment I am deleting most spam reports without reading. If you want to get my attention - especially if you discover a new trend or find some4thhing embedded in a thread, please send me a u2u.

[edit] formatting

[Edited on 31-12-2018 by j_sum1]

XeonTheMGPony - 31-12-2018 at 05:41

Why not automatically lock all inactive accounts, ie if some one makes an account and i has been inactive for 3 weeks lock posting privileges until they pm a mod to get it back?

Ubya - 31-12-2018 at 08:02

Why not automatically lock all inactive accounts, ie if some one makes an account and i has been inactive for 3 weeks lock posting privileges until they pm a mod to get it back?

mods already have a lot of work, plus it would be pretty boring to wait from a few hours to a few days just to comment something on the board (not everyone is active every day)

S.C. Wack - 31-12-2018 at 09:50

inactive for 3 weeks lock posting privileges until they pm a mod to get it back?

Brilliant...since that's the majority of actual members, why not lock all accounts then? Much easier than disabling new shitposters/spammers/personalities er members.

j_sum1 - 31-12-2018 at 13:07

Quote: Originally posted by S.C. Wack

inactive for 3 weeks lock posting privileges until they pm a mod to get it back?

Brilliant...since that's the majority of actual members, why not lock all accounts then? Much easier than disabling new shitposters/spammers/personalities er members.

No shortage of good ideas. But the active mods/admins are limited to solutions that do not require re-coding of the board software. What Melgar has achieved with minimal privileges is remarkable. Migration to a new platform is by far the best solution and will not require the kind of restrictions you are proposing.

S.C. Wack - 31-12-2018 at 16:24

The only thing I have proposed is disabling registration, a suboptimal here and now thing, not a wished for (wouldn't it be embarrassing if spam continued at a new board) future. I do wonder if public search engine visibility has caused more problems than it's worth.

j_sum1 - 31-12-2018 at 17:02

You are right. Disabling registration is suboptimal. We did it once for 2 weeks and, although the spam nearly stopped it was not welcoming for newbies. It was not sustainable.

Mods have limited options. Admins have more. The changes needed require back-end access which needs input from the board's owner.

Right now there are a few on holiday and so fixing things is slower. Sorry.

hacker - 31-12-2018 at 17:29

Quote: Originally posted by S.C. Wack

inactive for 3 weeks lock posting privileges until they pm a mod to get it back?

Brilliant...since that's the majority of actual members, why not lock all accounts then? Much easier than disabling new shitposters/spammers/personalities er members.

Why does Melgar have minimal privileges?

I can eliminate 99.9% of spam with one line of Python or PHP. Anyone can.

j_sum1 - 31-12-2018 at 17:47

You have already stated that you are a coding genius. Great. I am sure your one line of code is awesome.

Believe me. The mods and admins have looked at a number of options. But this is an old, inflexible system.

hacker - 31-12-2018 at 18:04

You have already stated that you are a coding genius. Great. I am sure your one line of code is awesome.

Believe me. The mods and admins have looked at a number of options. But this is an old, inflexible system.

That's beside the point. You didn't answer the question. Is Polverone MIA?

ninhydric1 - 31-12-2018 at 18:14

Yes, Polverone hasn't been on the forum for one and a half months.

fusso - 1-1-2019 at 02:32

Sadly Polverone isn't doing anything. I wonder if he simply wants to abandon the board?

hacker - 1-1-2019 at 03:36

The spambot is pretty weak. Or is this good?

Ubya - 1-1-2019 at 04:04

BotKilla probably died... too many spam threads

j_sum1 - 1-1-2019 at 04:44

This spammer has an old registration. Therefore it it not autodeleting the account when reported. Also, probably for a similar reason, it is not being flagged via botkilla's filters. I think the settings need tweaking.

ATM, this account is producing a string of 14 spam threads about 6-8 times a day. It also bypasses flood protection: posting at about 25 second interval. This means it is not a dumb bit of code. It is also adapting: albeit not as fast as another current spammer.

I have manually deleted the 14 threads maybe a dozen times over the past couple of days. It requires either an adjustment to Melgar's script or an admin to freeze the account.

hacker - 1-1-2019 at 04:53

Is this board run by adults?

WGTR - 1-1-2019 at 06:44

Yes, but the effective number of adults (ENA) who run the board who are present at any given time can be represented approximately by dividing the number of adults who run the board by 365, and then multiplying by the number of days that they are actually here. To achieve better granularity, it is better to improve resolution to an hourly basis instead of daily. Also, this doesn't account for the holidays (like right now), when most people are taking a much-needed break. Ha, ha.

Anyway, j_sum1, can you delete user accounts? Or change account passwords? One trick that I would use would be to re-register the account name and then retire it with a random password and a snarky message. I'm not sure what all you can do as a Super Mod.

See what I just did here?

MariaCoosy1 - 1-1-2019 at 07:04

MariaCoosy1 is an evil spammer who has been retired by WGTR. She is a filthy creature who has been condemned to reside trapped within the confines of an underserviced cat litter box.

fusso - 1-1-2019 at 08:33

@WGTR I see what you did down there

. I remember a mod had said that they can't delete an account. Not sure for changing passwords though.

j_sum1 - 1-1-2019 at 13:21

Right.

I just deleted 153 threads on weight loss by rarriedgex, posted over a period of 4 hours. Also about a dozen other spam threads and about 50 individual spam posts on existing threads.

What's that saying about doing six impossible things before breakfast?

Sorry that the forum has been a bit cluttered. Normal service will resume soon I hope.

And, ftr, I cannot change/delete accounts. Just posts/threads.

XeonTheMGPony - 2-1-2019 at 12:44

Is there no one that can ban that obnoxious bot? All most any post made is been spam as of late.

And second some one is available you mods need to be given the ability to remove accounts as this is rapidly making this forum near unusable. And that is a tragedy!

XeonTheMGPony - 2-1-2019 at 12:56

Quote: Originally posted by hacker

I can eliminate 99.9% of spam with one line of Python or PHP. Anyone can.

Then why the Waite? Script that sh*t and post it here for them !

and flood protection seems to be able to block me from posting, that is past the point of irony <_<

MrHomeScientist - 2-1-2019 at 13:14

rarriedgex is back up to 59 posts of spam. Where is everybody? Did the anti-spam bot die? Perhaps more mods should be granted account banning powers.

Right after posting the above, I had the pleasure of watching those posts disappear one by one. It was fun to keep refreshing the page and seeing his post count decrease every time.

Thanks!

[Edited on 1-2-2019 by MrHomeScientist]

j_sum1 - 2-1-2019 at 13:34

Seems I am the only mod on deck at the moment. It is getting lonely.

And yes, I just deleted another 70 rarriedgex threads and 28 other spam. Botkilla is still active and knocking off sp threads on average one per five mins.

MrHomeScientist - 2-1-2019 at 14:01

Annnnd it's back lol

When will the madness end?? Oh, the humanity!

Mr. Rogers - 2-1-2019 at 15:33

Put a CAPTCHA in the sign up.

Just curious, is this site running on ColdFusion?

[Edited on 2-1-2019 by Mr. Rogers]

Mr. Rogers - 2-1-2019 at 16:22

I appreciate this site immensely, and all the efforts of the staff and moderators here. I don't pay a dime for any of this, and I get that, and I'm grateful.

Right now, there's some Russian Roulette being played with a Glock. The Spam here isn't to sell anybody anything. When the bullet hits the patient is going to be dead.

WGTR - 2-1-2019 at 17:08

On an unrelated side note Mr Rogers, how are things going with you and your neighborhood? It’s a beautiful day isn’t it? A beautiful day in the neighborhood?

I suppose things will be looking up in the SM neighborhood once several mods and admins come back from vacation.

[Edited on 1-3-2019 by WGTR]

Mr. Rogers - 2-1-2019 at 17:29

Quote: Originally posted by WGTR

On an unrelated side note Mr Rogers, how are things going with you and your neighborhood? It’s a beautiful day isn’t it? A beautiful day in the neighborhood?

I suppose things will be looking up in the SM neighborhood once several mods and admins come back from vacation.

[Edited on 1-3-2019 by WGTR]

It's always a beautiful day in the neighborhood and I hope everyone enjoys their vacation.

[Edited on 3-1-2019 by Mr. Rogers]

Sometimes this spam thing is good for giggles.

j_sum1 - 3-1-2019 at 04:35

I came across this while deleting spam earlier today. I thought it was worth sharing.
Made me laugh anyway.

WGTR - 3-1-2019 at 07:34

In the United States we don’t tax most basic unprepared foods, but we do tax things like sodas, pet food, and so on. As a cashier, many years ago, I had a customer argue that she shouldn’t be taxed for dog food, because she eats it herself. You meet all sorts of interesting people in health food stores.

fusso - 3-1-2019 at 08:10

I came across this while deleting spam earlier today. I thought it was worth sharing.
Made me laugh anyway.

Maybe this ad is intended for vegan pet owners?

MrHomeScientist - 31-1-2019 at 10:10

I'm sure this has already been suggested at some point, but does the forum software allow filtering by keyword on new posts? Auto deleting anything with "porn" or "galleries" would get rid of a large chunk of spam, as a start. It's frustrating to see an old thread resurrected, get excited about new work, then be let down by spam.

fusso - 31-1-2019 at 10:54

Quote: Originally posted by MrHomeScientist

I'm sure this has already been suggested at some point, but does the forum software allow filtering by keyword on new posts? Auto deleting anything with "porn" or "galleries" would get rid of a large chunk of spam, as a start. It's frustrating to see an old thread resurrected, get excited about new work, then be let down by spam.

I remember Melgar made a spam threshold for the spam filter but don't understand why recent spams stayed up longer despite having similar content.

Ubya - 31-1-2019 at 12:05

"everyday chemistry (provisional)" is getting so spammed we can't keep up, same story with many sticky threads

j_sum1 - 3-2-2019 at 14:27

Can I swear?

#&&$%#**!

I have ben away for 24 hours and came back to 130 spam reports and more than 30 threads with embedded spam. And it looks like another mod had been around to clean up the spam threads.
If you think something got missed then send me a U2U. I am not going to open and read all the spam reports.

fusso - 3-2-2019 at 16:12

Seriously, when will Polverone come back to do the forum migration stuff? Just finish this step and we'll be free from spams again :mad:

Jackson - 4-2-2019 at 09:47

One thing I’ve noticed is that spam is now being posted as replies to already existing threads. Is there a script to deal with this? If not I think one should be implemented.

j_sum1 - 4-2-2019 at 13:47

Quote: Originally posted by Jackson

One thing I’ve noticed is that spam is now being posted as replies to already existing threads. Is there a script to deal with this? If not I think one should be implemented.

That's called survivor bias.
I doubt the proportion of new-thread to appended spam has changed much. It is just that the spam has increased a whole lot. And we are catching more than ever before.

A script for dealing with spam appended to threads might be a good idea but it carries greater risk of deleting legitimate posts: especially from newbies.

Tsjerk - 4-2-2019 at 14:32

That's called survivor bias.
I doubt the proportion of new-thread to appended spam has changed much. It is just that the spam has increased a whole lot. And we are catching more than ever

That is a very nice thing, for you guys to do, not for the forum. But you guys are doing a very nice, much underestimated effort.thank you so much and lets hope we can do the switch soon.

Tsjerk - 4-2-2019 at 14:40

If Polverone doesn't return soon, can't we transfer the site to a new ip / database but without the u2u's? As far as I know we only need P for the u2u's.

People can back u2u's up themselfs or could retrieve them from a website which has a blocked posting functionality. I think there are members on this site who could block all posting.

Bad idea?

Ubya - 5-2-2019 at 11:24

botKilla is not working at the moment, the board is full of spam

SPAM, spam, spam and more fucking spam...

hissingnoise - 5-2-2019 at 12:50

Jeeez, is there no one who can stop the spammers turning this site into utter rubbish???

j_sum1 - 5-2-2019 at 13:08

It was clean 6 hour ago. I can do another clean up in an hour. Thanks for your patience.

And I agree: it is a bit ridiculous the voume arriving here.

j_sum1 - 5-2-2019 at 13:58

It seems that botkilla is down.
I will do my best to keep things clean until Melgar restores it.

fusso - 5-2-2019 at 14:05

jsum just became admin?!:O

j_sum1 - 5-2-2019 at 14:11

jsum just became admin?!:O

And texium too.

I am enjoying my new superpowers.

fusso - 5-2-2019 at 14:18

jsum, could you please delete the spam accounts completely when you see them start spamming the first few posts? Just removing the spam posts doen't seem to be enough.

j_sum1 - 5-2-2019 at 14:32

jsum, could you please delete the spam accounts completely when you see them start spamming the first few posts? Just removing the spam posts doen't seem to be enough.

Like I said, I am enjoying the new superpowers.
There are spambots that run on different algorithms. Most are single-post nasties, so no real benefit in deleting the accounts. The return spammers get silenced: a few different strategies depending on characteristics.

RogueRose - 6-2-2019 at 10:09

Is there no way to ban every poster who makes a post with the word "porn", sexy, tits, free sex?

I just reported about 30 posts and over 3/4 had one of those words in it. If the poster was banned as soon as they posted one of those words, all their other posts could be deleted as well.

Also, is there a way to copy all URL's from each spam post and put them in a large black-list and run a check for any post with URL's against the blacklist. Then if any of them appear, ban that user and delete all their posts. I've seen SO many posts that link to the same site(s).

Tsjerk - 6-2-2019 at 10:25

I think that would work, but it is like suiting the disease, while there is a cure. Transfer of the forum to new software would fix stuff and and as far as I know Melgar could be ready and set to go.

andy1988 - 6-2-2019 at 10:36

Does the restriction manager work at all?

Ban all new user registrations, EDIT: use wildcards on email/username field? *@*
Make a sticky thread with a HOWTO get a new account
Directions include emailing an email address with a certain subject, filter out all emails that don't match said subject
Email body/subject includes an answer to a simple chemistry quiz (e.g. what is the most basic pH possible? What is above Ag in the periodic table?).
Email body includes desired username(s) (more than one in case a username was taken)
Admin/script can create new account manually with temp password? Less work this way than manually deleting spam that makes it past botkilla?

I see something here too about a setting to make a first post having to be approved before it's visible. But the above sounds like less work.

Could also make a script only allowing new account creation on a certain timeframe, e.g. every 12 hours for 10 minutes, e.g. 1200UTC-1210UTC, 0000UTC-0010UTC. Would be a little inconvenient but no barrier to a person seeking insight with some problem they're working on anyway.

------------------

Google lowers the rank of websites taken over by spam, I don't recall where I learned this, it may have been a presentation by a Google Engineer at my alma mater. Their algorithms can see through these primitive attacks, all in order to keep their search results high quality!
e.g. search term "sodium metal laboratory electrolysis" showing a sciencemadness.org result may move from page 1 to page 2 or beyond, and etcetera for all pages.
EDIT: I guess their attacks may manipulate more primitive search engines like duckduckgo or vk. Wouldn't fool Google though... I'm kind of thinking vk given the language often used.

One of the first jobs in the USA I had the opportunity to take was $7/hr (way more than $5.15 minimum at the time!), writing essays to increase the search rank of certain niche marketplace websites (essays on said products/website stored on different domains clustered appropriately). Before Amazon got big things were more competitive. They had a room with a handful of young American women writing essays on shifts, and wanted me to write C code to script things. They made me sign an NDA to even talk about the job details... whoops!

One other annoying thing you see in most website forums is the insistence of creating a new thread for every new user's problem- this choice of rule is meant to maximize ad revenue to the detriment of finding optimal solutions to problems (spaghetti-ing observations and solutions all over the place in threads making use of volunteer labor). There was a nice article which generalized forum politics like this, I wish I'd bookmarked it.

[Edited on 7-2-2019 by andy1988]

Spam

Predacon35 - 6-2-2019 at 12:50

Some of these spam posts seem to get quite a few views-

unionised - 6-2-2019 at 13:01

Quote: Originally posted by Predacon35

Some of these spam posts seem to get quite a few views-

That's because opening them to report them as spam gets counted as viewing.

j_sum1 - 6-2-2019 at 13:49

The problem at the moment is that botkilla is down. Melgar needs to get it up and running -- which I am sure he will do when he next gets an opportunity.
Botkilla was only ever a temporary measure. A migration to a new platform is indeed the way to go.

Restriction manager does work but it is not a particularly powerful instrument as it targets usernames only. (AFAIK)
Your other suggestion andy might have to be the way that we go.

It is noticeable that the spam patterns adapt pretty quickly. A week ago the pattern was to add 14 spam posts to an existing thread. That does not seem to be happening at the moment.

I am off to do a clean-up.

Tsjerk - 6-2-2019 at 23:37

I am off to do a clean-up.

Thanks!

Please fix security or close new registrations.

Sulaiman - 16-2-2019 at 06:21

SM is not in urgent need of new members

closing new registrations until security works would protect existing members from all of the spam

a notice explaining the above in place of the registration should passify any potential new members until this can be fixed.

Please fix security or close new registrations until the new software/site is working properly.

j_sum1 - 16-2-2019 at 18:42

This time I disagree with you, Sulaiman.

Members, both old and new are the lifeblood of SM. Last time registration was closed for two weeks we heard through the grapevine of a number of people who wanted to sign up but couldn't. We do not want to turn these people away. Who knows how many just gave up.

The contribution that newbies make is valuable since they are the ones that ask questions and promote discussion. And we need also to have new members to compensate for the seasoned members whose lives turn another direction and who cannot make contributions as regularly (if at all).

WRT to the spam... progress is being made. We all know that the final solution requires Polverone's input. Jesus may return before that happens. So in the meantime we have adopted secondary measures with the fairly limited tools we have.

Added to the fun is that botkilla, the script put together by Melgar, has been down for a few weeks. Botkilla was catching spam more frequently than once every five minutes on average. Do the Math: that is in the order of 300 spam posts per day. Currently, we are about a tenth of that -- I have been able to block spammer registration to some extent and I am dealing with roughly 20-30 spam posts per day at present. And for every spam account I find, I search out related sleeper accounts that have been registered in the past.

If you see a spammer -- or more significantly, if you see a spammer disappear, there is a chance that I have neutralised somewhere between 2 and 150 spammer accounts. Sure it is a manual process rather than an automated one. But it has been effective. I know the account characteristics that are currently slipping through the registration net. They are a bit trickier to filter. But there are yet a couple of tools that I can implement. They will just require an investment of time.

By my estimate, once botkilla is up again, we will only see one or two spam posts per day. I was a bit worried a few weeks back but now I am very optimistic.

And what is also very gratifying is the increased level of discussion and activity currently on the board. When the spam is gone, the people come back.

There is still a concentration of spam that tends to arrive during my sleep hours. If you notice it, try not to get too frustrated. It will be dealt with in due course.

fusso - 17-2-2019 at 14:14

Members, both old and new are the lifeblood of SM. Last time registration was closed for two weeks we heard through the grapevine of a number of people who wanted to sign up but couldn't. We do not want to turn these people away. Who knows how many just gave up.

The contribution that newbies make is valuable since they are the ones that ask questions and promote discussion. And we need also to have new members to compensate for the seasoned members whose lives turn another direction and who cannot make contributions as regularly (if at all).

A number of ppl who wanted to sign up but couldn't? How many? When did this happen? And how many real ppl register per day currently?

j_sum1 - 17-2-2019 at 14:24

A number of ppl who wanted to sign up but couldn't? How many? When did this happen? And how many real ppl register per day currently?

Last year some time. I forget exactly when. Not that it matters.
Probably one or two genuine sign-ups per day on average.

Take a look at https://www.sciencemadness.org/whisper/misc.php?action=onlin... and see how many "members" have visited in the last 24 hours. The majority of these are spam accounts that have signed up within the past two days and have been blocked from posting.
(edit) The advantage of allowing these without deleting the accounts is that many bots are dumb enough to not know that their posts fail. This means that they don't create new accounts.

Sure we could close registration and block these accounts completely, but in the big scheme of things, the situation is currently under control and genuine new members are unrestricted.

[Edited on 17-2-2019 by j_sum1]

Waffles SS - 19-2-2019 at 22:08

I didnt read whole of this topic but i suggest to use CAPTCHA verification for each post.(at last bot and script cant create or post topic)

j_sum1 - 19-2-2019 at 22:24

Waffles, that suggestion has come up a few times.

This is old software and the captch abilities are unsophisticated. The real solution is a new platform.

I intend to experiment with the captcha option such as it it: now that I have access to these things. But probably not for a week or two when I have some time to invest.

The good news is that even though I have been out of action for a couple of days, and other mid presence has been low, the board has not drowned under spam. Something is working.

j_sum1 - 25-3-2019 at 21:10

Thanks to all who have been reporting spam.
Today I empty the trash of spam reports going back a few months.
(My apologies for not replying to each one individually.

)

WangleSpong5000 - 25-3-2019 at 21:30

Spam if off the chain! Getting proper russian spam now... backwards R's and everything lol

froot - 25-3-2019 at 21:40

Which would be easier?

Restricting registrations to admin only who registers new members after receiving motivational email used to filter spam accounts. How many new valid registrations can we expect in a day?

Or

Do what you are doing now.

Sometimes easier is better.

j_sum1 - 25-3-2019 at 22:40

It's been a busy day with spam. A new spambot with a new pattern of behaviour. The good news is the deactivation of scores of spam accounts and some new registration filters.

Honestly, I don't know how well manual registrations would work. I think frustrating to newbies: particularly if admins are spending time living. I think it would also requre tweaking some settings that are above my pay grade. So I dunno.

All things considered, the situation is not dreadful at present. Certainly better than a couple of months ago.

WangleSpong5000 - 26-3-2019 at 00:37

It's really not so bad. Frustrating for you mods no doubt but there's not really very much of it...

fusso - 30-3-2019 at 19:46

When will Melgar's bot run again??? :mad:

j_sum1 - 30-3-2019 at 22:28