| Pages:
1
..
18
19
20
21
22
..
28 |
j_sum1
Administrator
Posts: 6320
Registered: 4-10-2014
Location: At home
Member Is Offline
Mood: Most of the ducks are in a row
|
|
There is at least one bot that hits all 14 forums. And for some reason is able to bypass the flood protection. Your post indicates two spam per
minute but it is sometimes an much as three.
There are some other similar spam that arise and only hit one forum -- usually Gen Chem. I think these ones get thwarted by flood protection.
|
|
|
Diachrynic
Hazard to Others
Posts: 226
Registered: 23-9-2017
Location: western spiral arm of the galaxy
Member Is Offline
Mood: zenosyne
|
|
Idea: what if we take the ten most common English words and run a quick check over a new post. If there is a high enough deviation (which can be as
much as only one word of that list in the entire post) it needs to be checked by a moderator first before it gets send. This should only apply to new
topics. The moderator then either decides to show it or to trash it.
Of course bots can easily circumvent this, but maybe there is some use to this idea.
The idea is also flawed in that it is not automatic. In order to prevent legitimate topics from being trashed a human being is needed. But the mods
could be completely overwhelmed by the number of posts.
Like I said, just something that came to my mind because I noticed the bots just shove a bunch of keywords into a post but almost never say
"I am trying to do the..." or something like that.
we apologize for the inconvenience
|
|
|
Swinfi2
Hazard to Others
Posts: 131
Registered: 19-2-2018
Location: England
Member Is Offline
Mood: Catalytic
|
|
@streety the "don't you like sex" thread. I read the first 2-3 pages and it looks like a compilation of everything to do with nerve agents/drugs and
other bad stuff that (when organised as such) kinda implicates the forum staff as they have posts amongst it.
I have a hard time believing the post is "real" as many posts reference poeple/posts that are missing and context jumps. It looks to me like an AI
word salad of coppied posts to give the appearance of coherance.
How could that even happen? That thread freaks me out, it looks like a set up imo.
|
|
|
fusso
International Hazard
Posts: 1922
Registered: 23-6-2017
Location: 4 ∥ universes ahead of you
Member Is Offline
|
|
Quote: Originally posted by Swinfi2  | @streety the "don't you like sex" thread. I read the first 2-3 pages and it looks like a compilation of everything to do with nerve agents/drugs and
other bad stuff that (when organised as such) kinda implicates the forum staff as they have posts amongst it.
I have a hard time believing the post is "real" as many posts reference poeple/posts that are missing and context jumps. It looks to me like an AI
word salad of coppied posts to give the appearance of coherance.
How could that even happen? That thread freaks me out, it looks like a set up imo. |
And I briefly skimmed a
few posts starting from the bottom. I noticed some spam posts at the bottom. Going up, the thread seemed to be detritused due to trolls. And I found a
few currently active members there. I hope someone who was involved could explain it for dummies.
|
|
|
j_sum1
Administrator
Posts: 6320
Registered: 4-10-2014
Location: At home
Member Is Offline
Mood: Most of the ducks are in a row
|
|
I don't recall the thread. U2u me a link. If it is in detritus then that might be too good a place for it. Sounds like nothing will be lost in killing
it.
|
|
|
fusso
International Hazard
Posts: 1922
Registered: 23-6-2017
Location: 4 ∥ universes ahead of you
Member Is Offline
|
|
@jsum well here you are.
|
|
|
j_sum1
Administrator
Posts: 6320
Registered: 4-10-2014
Location: At home
Member Is Offline
Mood: Most of the ducks are in a row
|
|
| Quote: Originally posted by fusso | | Quote: Originally posted by streety | | In putting together these figures I discovered two odd posts. The post date is 1969. In the database they are represented by timestamp values of 0 and
1. They are also clearly spam but then there are 8 pages of legitimate content. |
@jsum well here you are.
|
Ok. I took a look. It can stay in detritus. It actually looks like a collection of stuff from several different threads including trolling,
spamming and legitimate discussion of two or three different topics.
I don't know the thread's history -- it all seems weird. But it is not doing any harm where it is.
|
|
|
WGTR
National Hazard
Posts: 971
Registered: 29-9-2013
Location: Online
Member Is Offline
Mood: Outline
|
|
Negative 15 posts?
|
|
|
CharlieA
National Hazard
Posts: 646
Registered: 11-8-2015
Location: Missouri, USA
Member Is Offline
Mood: No Mood
|
|
Tonight is especially frustrating...3 pages of posts today, and 2 pages of them are spam. I think when a new account is made, the account shouldn't be
able to post for a week. There is much talk here about this subject but nothing seems to get done about it. I wish I were computer literate but I'm
not, so I'm unable to do anything about it (except leave the forum).
|
|
|
streety
Hazard to Others
Posts: 110
Registered: 14-5-2018
Member Is Offline
|
|
This is an update on my post from the top of page 19.
The script was updated and run every 2 minutes. I restarted it on the 19th and analyze only full days below, so from the 20th August to the 14th
September.
During that time I recorded 2803 spam topics for a rate of 100/day.
The histogram below shows the time it took to delete each spam topic.
The minimum was again barely above the sampling frequency of the script so I still probably missed some topics that were deleted so quickly they were
gone before my script downloaded the page. Should be less than last time though.
The average was 69 minutes(down from 84 minutes). 34% (up from 32%) were deleted within 30 minutes, 64% (up from 47%) within 60 minutes, and 83% (up
from 79%) within 120 minutes. The median was 45 minutes (down from 65 minutes).
There isn't much effect of time of day on spam duration in this period.
You may notice the topic deleted after more than 3000 minutes. It was not a spam post so I may need to think about how to handle deletions that are
not spam related. In this case it seems to be a legitimate member who created a new topic in error instead of a reply to an existing topic. It was
then later cleaned up by an administrator.
Yesterday was particularly bad for spam. Hopefully it won't be the start of an up-tick in the posting frequency.
|
|
|
JJay
International Hazard
Posts: 3440
Registered: 15-10-2015
Member Is Offline
|
|
The spammers are in rare form today. I have never seen so much. I must have reported 20 of them this morning.
|
|
|
WGTR
National Hazard
Posts: 971
Registered: 29-9-2013
Location: Online
Member Is Offline
Mood: Outline
|
|
I would suggest that Polverone add some extra people to the trusted reporters list. When I check in I often find spam that's been here for 6 hours or
more, with a couple dozen views and presumably at least half that many reports. Then when I report it, the thread is gone within a minute or two.
That tells me that the people doing most of the reporting aren't "trusted reporters" yet. Keep reporting it though, because Polverone keeps an eye on
who's doing the reporting, and sometimes adds additional people to the "trusted" list. Also, when I log in I can clear out several pages of spam at
one go if others have already reported it.
|
|
|
streety
Hazard to Others
Posts: 110
Registered: 14-5-2018
Member Is Offline
|
|
The past few days have been quite bad but today doesn't seem extraordinary.
The median period of time spam posts are hanging around was a little higher than usual.
I have been working with woelen to deploy a script to help with the spam. I should really follow up on that. Essentially it would do exactly as you
suggest and increase the impact more members could have.
Edit to include most recent day in duration plot. Changing the interpretation.
[Edited on 22-9-2018 by streety]
[Edited on 22-9-2018 by streety]
|
|
|
symboom
International Hazard
Posts: 1143
Registered: 11-11-2010
Location: Wrongplanet
Member Is Offline
Mood: Doing science while it is still legal since 2010
|
|
Anyone know if they are using AI (artificial intelligence) yet
Got to be ahead of the game. Divergent and convergent thinking
Open message
To streety
This guy lesterpq11 look at for pattern see key words to outwit the AI
AI seems to try to imitate humans where ever it gets the info from
[Edited on 22-9-2018 by symboom]
|
|
|
Melgar
Anti-Spam Agent
Posts: 2004
Registered: 23-2-2010
Location: Connecticut
Member Is Offline
Mood: Estrified
|
|
Oh, I did some analysis on the test database. Turns out 99% of our users have 0 posts. Most of those registrations were automatic, and had names
that had obviously generated automatically. It seems our active membership is perhaps around 8000.
By the way, my efforts to get this site working using phpBB has had quite a few breakthroughs in the last week or so. Check it out:
http://35.185.63.230/talk/index.php
The theme is just a random one, and other themes can be added and used. There are some bbcodes that I've had difficulty converting, and there's
definitely some more things that would need working out, but the hardest parts are all behind me now. Thoughts?
[Edited on 9/22/18 by Melgar]
The first step in the process of learning something is admitting that you don't know it already.
I'm givin' the spam shields max power at full warp, but they just dinna have the power! We're gonna have to evacuate to new forum software!
|
|
|
fusso
International Hazard
Posts: 1922
Registered: 23-6-2017
Location: 4 ∥ universes ahead of you
Member Is Offline
|
|
| Quote: Originally posted by Melgar | By the way, my efforts to get this site working using phpBB has had quite a few breakthroughs in the last week or so. Check it out:
http://35.185.63.230/talk/index.php
The theme is just a random one, and other themes can be added and used. There are some bbcodes that I've had difficulty converting, and there's
definitely some more things that would need working out, but the hardest parts are all behind me now. Thoughts?[Edited on 9/22/18 by Melgar]
|
Why is the last post in 2016 not something more recently (eg this year)?
[Edited on 22/09/18 by fusso]
|
|
|
Melgar
Anti-Spam Agent
Posts: 2004
Registered: 23-2-2010
Location: Connecticut
Member Is Offline
Mood: Estrified
|
|
Because I'm working with an older database backup that was stripped of personal information by Polverone. The idea was to write the code to do it on
the older database backup as a proof of concept, then use the same or similar code to convert the live database when we're ready to do that.
There have been pushes to do this for years now, and migrating would certainly solve the spam problem.
The first step in the process of learning something is admitting that you don't know it already.
I'm givin' the spam shields max power at full warp, but they just dinna have the power! We're gonna have to evacuate to new forum software!
|
|
|
RogueRose
International Hazard
Posts: 1593
Registered: 16-6-2014
Member Is Offline
|
|
This may have been asked and stated but I haven't seen it. How many reports of spam are needed for all the users posts to be deleted?
What I want to know is if I see 10 posts by a user, how many need to be reported for the program to sweep up all their posts?
|
|
|
CuReUS
National Hazard
Posts: 928
Registered: 9-9-2014
Member Is Offline
Mood: No Mood
|
|
anti spam measures - https://web.archive.org/web/20151125135503/http://www.ninjap...
distilled from wiki-https://en.wikipedia.org/wiki/Forum_spam#Spam_prevention
| Quote: | 1.Blacklisting services such as fspamlist, StopForumSpam and keep databases of IP addresses, usernames and e-mail addresses used to post spam or
register forum accounts.Forum software can query these lists and either deny posts or registration, or submit the request for human moderation. This
is similar to DNSBL services.
2.Simple CAPTCHA systems which display alphanumeric characters have proven vulnerable to optical character recognition software but those that
scramble the characters appear to be far more effective
3.Textual confirmation,in which the user answers one or more random questions to prove that he/she is not a spambot - ( doesn't have to be chemistry
questions,could be questions like "how many letters in sciencemadness" or in which language is it ?)
4.Confirmation e-mails to registering users prior to allowing the user a first log in, either containing a site-generated password or an activation
code/link
5.Authoritative voice, using an external filtering service to get a verdict if the data is spam or not.( free filtering services available)
6.Denial of registration from certain domains that are a major source of spambots, or even domain extensions such as .ru, .br, .biz
7.Using a search engine to investigate usernames for hits as recognized spambots on other forums(this could be coded into SM so that it does it
automatically.)
8.Changing technical details of the forum software to confuse bots — for example, changing "agreed=true" to "mode=agreed" in the registration page
of phpBB
9.Blocking posts or registrations that contain certain blacklisted words ( can be automated)
10.A useful technique for proactive detection of well-known spammer proxies is to query a search engine for this IP. It will show up on pages that
specialize in the listing of proxies.( again can be automated)
11.Redirecting spammers to "spam subforums" to direct spam away from human users on the main site( or even do a return to sender approach and beat
them at their own game  ) |
My own suggestions-
1.Most spammers have a link in their message,so we could detect posts with links and block them(for 1st post only)
2.Do not allow newly registered members to post more than 1 message or in more than 1 sub forum.
3.Block usernames or posts with non english alphabets
someone had posted an amazing idea is this thread,but I can't seem to find that post now.The idea was to run usernames through a password strength
checker.Since bots use long alphanumerical strings,they would indirectly make very strong passwords,which could be detected and blocked.
We must do something fast,or pretty soon we would have to build another arc to escape this flood
[Edited on 23-9-2018 by CuReUS]
|
|
|
RogueRose
International Hazard
Posts: 1593
Registered: 16-6-2014
Member Is Offline
|
|
I have a feeling that there is more going on than just spam. I have a feeling that the posts might be a way to pass messages to others with no record
of them. The posts are up for a short time and then the "system" erases them. While they are up they are grabbed. I would suspect that the spam
bots wouldn't continue to post here if it wasn't getting some kind of return. It wouldn't post with such furry unless there was a benefit being had
and I don't think it is members buying access to adult sites or ED pills.
This could be a serious issue that really needs taken care of and is wreckless allowing it to continue.
The mods should "deputize" some of the members to allow them to review the first posts of new members, all new posts get held until "OK'd" by a
"deputy SPAM bot". The member checks a hidden sub forum where the new posts are sent and clears good posts. This would also work well to block
people from signing up to reply to 9 year old threads applauding a cook and bashing someone protecting the forum.
|
|
|
Melgar
Anti-Spam Agent
Posts: 2004
Registered: 23-2-2010
Location: Connecticut
Member Is Offline
Mood: Estrified
|
|
It's just bots that scan the internet for message boards that they're able to automatically register at. They use various tactics to try and make it
harder to keep them out. Some of these tactics probably don't make any logical sense, but they kind of operate on a "throw a bunch of shit at the
wall and see what sticks" philosophy. The only real way out is to migrate to software that's still being actively developed, which I'm trying to do
with phpBB. If anyone has any thoughts on this, I'd love to hear to them. If you want server access, send me your RSA public keys, and I'll add you
to authorized_hosts. If you want admin access on the phpBB test site, register at the link I posted above, and message me your username, and I'll
make you an admin. If you want to look at the forum databases, I'm running a virtual machine of the XMB server within the phpBB server, and you can
access both MySQL databases via TCP/IP if you're logged in. I have it set up so I can easily pull data in from both databases via a Ruby console.
Like, to show data for a random user, you can type XMB.members.random, or PHPBB.users.random, respectively. I would really like some help with the
PHP part, since my own PHP skills are severely lacking. I've gotten this far mainly by using Ruby and relying on my decent SQL background. Most of
the rest of what's left is just annoyances. Like the fact that [size] and [attachment] tags work differently in phpBB. I'm not sure whether to try
and script the conversion of XMB tags to their phpBB versions (something I did already with the [rquote] tags) or try and make the XMB bbcode tags
work as they are.
Right now, the goal is to get it to a point where we like how it looks, then set up a new board with all the settings imported from the one I'm
working on now. Presumably run by Polverone. Then use the tools we've developed to transfer the data over.
XMB hasn't been actively developed since 2009, and I'm pretty sure we had a consensus a long time ago that we're going to have to transition to new
software if we ever want to address this problem. Correct me if I'm wrong.
The first step in the process of learning something is admitting that you don't know it already.
I'm givin' the spam shields max power at full warp, but they just dinna have the power! We're gonna have to evacuate to new forum software!
|
|
|
RogueRose
International Hazard
Posts: 1593
Registered: 16-6-2014
Member Is Offline
|
|
| Quote: Originally posted by Melgar | It's just bots that scan the internet for message boards that they're able to automatically register at. They use various tactics to try and make it
harder to keep them out. Some of these tactics probably don't make any logical sense, but they kind of operate on a "throw a bunch of shit at the
wall and see what sticks" philosophy. The only real way out is to migrate to software that's still being actively developed, which I'm trying to do
with phpBB. If anyone has any thoughts on this, I'd love to hear to them. If you want server access, send me your RSA public keys, and I'll add you
to authorized_hosts. If you want admin access on the phpBB test site, register at the link I posted above, and message me your username, and I'll
make you an admin. If you want to look at the forum databases, I'm running a virtual machine of the XMB server within the phpBB server, and you can
access both MySQL databases via TCP/IP if you're logged in. I have it set up so I can easily pull data in from both databases via a Ruby console.
Like, to show data for a random user, you can type XMB.members.random, or PHPBB.users.random, respectively. I would really like some help with the
PHP part, since my own PHP skills are severely lacking. I've gotten this far mainly by using Ruby and relying on my decent SQL background. Most of
the rest of what's left is just annoyances. Like the fact that [size] and [attachment] tags work differently in phpBB. I'm not sure whether to try
and script the conversion of XMB tags to their phpBB versions (something I did already with the [rquote] tags) or try and make the XMB bbcode tags
work as they are.
Right now, the goal is to get it to a point where we like how it looks, then set up a new board with all the settings imported from the one I'm
working on now. Presumably run by Polverone. Then use the tools we've developed to transfer the data over.
XMB hasn't been actively developed since 2009, and I'm pretty sure we had a consensus a long time ago that we're going to have to transition to new
software if we ever want to address this problem. Correct me if I'm wrong. |
Yes, I know very well how SPAM bots work in a normal manner, but what I'm saying is that how can you ever tell what it is doing? I've never seen spam
posted like on this board, anywhere, where 5,000 character posts are repeatedly posted day after day with new (or slightly altered) content. To me,
it looks like coded messages and unless you've dealt with that before, I doubt you would understand what to look for - and THAT IS what it looks like
to me.
If there is anything I or anyone else can do to help, let us know
|
|
|
WGTR
National Hazard
Posts: 971
Registered: 29-9-2013
Location: Online
Member Is Offline
Mood: Outline
|
|
I just went to send a U2U, and was informed that I have to delete some messages, as I've exceeded my limit of 5000. I have something like 5,600
messages in my outbox.
Apparently when I report spam, a message gets sent to every admin, every time...and they don't get automatically deleted.
Now, since I have a few dozen sent messages that I'd like to keep, I have to go into my outbox and manually look through all 5,600 of these spam
reports so that I don't accidentally delete something important. Sigh.
|
|
|
j_sum1
Administrator
Posts: 6320
Registered: 4-10-2014
Location: At home
Member Is Offline
Mood: Most of the ducks are in a row
|
|
| Quote: Originally posted by WGTR | I just went to send a U2U, and was informed that I have to delete some messages, as I've exceeded my limit of 5000. I have something like 5,600
messages in my outbox.
Apparently when I report spam, a message gets sent to every admin, every time...and they don't get automatically deleted.
Now, since I have a few dozen sent messages that I'd like to keep, I have to go into my outbox and manually look through all 5,600 of these spam
reports so that I don't accidentally delete something important. Sigh. |
Set up a new folder for U2Us you want to keep. Scroll through and click everything that is not a spam report and then move to your new folder. Once
done you can delete everything else.
It is a bit of a pain but it won't actually take too long. I did the same thing when I took on the mod role.
|
|
|
WGTR
National Hazard
Posts: 971
Registered: 29-9-2013
Location: Online
Member Is Offline
Mood: Outline
|
|
Huh. I didn't know you could add folders like that, but I figured it out. Thanks! It looks like that's exactly what I needed.
|
|
|
| Pages:
1
..
18
19
20
21
22
..
28 |
![[*]](images/xpblue/default_icon.gif){kind=icon}
