I have uploaded an encrypted backup of the sciencemadness forum database to axehandle's FTP site. The file name is
/upload/SM-backup-08052004-xmb1.gz.asc.gz. In the event that the site should suffer some grave failure, I might have to rely on members who downloaded
this file to supply me with it again so that the board can be restored. I will give the private key to my fellow moderators as well, so that there is
no single point of failure that could mean the irrecoverable loss of the information in our database.
I trust that the FTP account-holding members here will do their part to make sure this archive is widely distributed. If this works out well, I will
place my future backups (encrypted, of course) on the FTP site as well.
Edit: all moderators, will you please post your PGP public keys in this thread? I don't want to send the private key to the archive as plaintext.
[Edited on 8-5-2004 by Polverone]jimmyboy - 5-8-2004 at 18:02
could this be done as well for roguesci on their ftp - or is it their already?Hang-Man - 5-8-2004 at 18:18
Can't be done unless somone has the backups, or feels like running through all the google caches (I found a few gems) The Rouge Sci ftp is still
up, which is a bit odd.....Sarevok - 5-8-2004 at 19:36
Polverone, can the backup be made available for those who don't have FTP access? My account was deleted, for a reason.
The more people with the backup, the better.Eliteforum - 6-8-2004 at 02:39
What's the chances of someone cracking the file and getting everyones passwords?
Lol!
Quantum - 6-8-2004 at 04:40
If someone can crack PGP then cracking forum passwords would be like taking
C<sub>12</sub>H<sub>22</sub>O<sub>11</sub> from an infant!Saerynide - 6-8-2004 at 08:15
No one can crack PGP, but we don't know what the NSA is hiding now, do we? I know what Im talking about is almost impossible, but if
anyone's read Digital Fortress, or stuff about quantum computers, they'll know what Im talking about. Who knows? Maybe the governments
already have broken PGP... :S
Im not trying to make anyone paranoid here, k? I dont believe anyone has broken PGP Polverone - 6-8-2004 at 09:23
The database contains only password hashes, not the passwords themselves. Still, the hashes can be valuable. When the mods have provided me with
public keys so that I can send the private key, they will all have access (if they so choose) to the password hashes used here. Nobody here is so
careless as to use their sciencemadness password for any other system or service... right?
I don't think any private citizen is going to be able to crack the encryption on this file. It would of course be trivial if they private key
were stolen. Keeping the key on removable media or on a non-net-connected machine might be a good idea, if we want to put on our worst-case-scenario
thinking caps.JC - 6-8-2004 at 14:24
Well, the FTP is at a completely different IP than the server that hosted the site.
As for the password hashes, well, they technically can't be reversed. However, you can take the possible words and run them through the same
encryption/hashing algorithm, then compare them. This takes a while, as hashes are designed to be slow, so savvy hackers tend to have large databases
of hashes of common passwords. This is then queried via SQL, and the answer found. Not much of a risk if you have a strong password, but most people
don't bother for forums.
Also, the NSA could have a db of billions of rows, each word having an MD5, blowfish, SSL, etc. equivalent. It would be hard to build, but very useful
on a daily basis.
As for breaking the PGP key, etc., well, there are plenty of people who would love to be able to. Currently, the cost of breaking a key is too high.
However, I have a friend who is convinced that the US government have the ability. I was pretty sceptical, but he tried to convince me. I came up with
a way to reduce the scale of the crack by about log n, and he came up with a feasible way to break a password in a human scale time. Combine the two,
and you could get into a short PGP key in little time. Of course, each system would cost millions, but what else do the NSA do all day? They will have
the kind of processing power that will be self-aware as soon as someone works out how to write the code! In short, I wouldn't trust any current
system to keep me in the clear for more than five years, since that's about how long it would take to break it. The best insurance is not to be a
priority job! Osama's posts get 99% of it, and are cracked in a few months/years, I am sure. My grocery orders aren't a part of that
priority effort.
[Edited on 6-8-2004 by JC]vulture - 7-8-2004 at 06:42
Quote:
My grocery orders aren't a part of that priority effort.
No, but if you're trying to get into someones comms, you're cracking everything. So send alot of PGP messages which don't make alot of
sense, use some gibberish code language, etc and drive the blokes at the NSA (do they ever get sunlight?) to madness.
Backup
MadHatter - 7-8-2004 at 10:00
I have lots of hard drive space and a cable connection to the
net. What is the fastest way to backup this forum ?
Ref: Way to extract a web site.....
solo - 2-1-2005 at 20:10
Look for a program that is a web extractor ....look in goggle......solochemoleo - 2-1-2005 at 20:23
Yes, I tried this before after the down-period, using various web zip programs from the net. In fact, I tried pretty much every single one I could
find.
It doesnt work well though. You can indeed download the threads, but it is of course just an html file, and despite all my efforts I did not manage to
exclude links that are repetitive, such as 'post reply', 'quote' etc. Therefore one accumulates a vast amount of useless htmls. As
a result, I stopped after I downloaded 200 MB, for I didnt want to stress the web host... that is, Polverone
Having worked with Perl script elsewhere, I figure this is the only way to download a forum site properly, without accumulating lots of junk.
But I am sure there are plenty of people who know much more about htis than I do, care to comment? S.C. Wack - 2-1-2005 at 20:25
Firefox plugin Spiderzilla works very well for me, though I've never done it here. Might there be spidering defenses on this board?
[Edited on 3-1-2005 by S.C. Wack]Polverone - 2-1-2005 at 20:48
It is difficult to properly automate archival of web-forums due to the use of passwords and (as you mentioned) redundant links. Simplistic traversal
will yield many worthless document duplicates. I think the only practical way to do this is to write a custom program specifically designed to archive
a single web forum or class of web forums (like XMB forums, YaBB forums, etc.) It's fairly easy to do this in a straightforward manner, harder if
you want to add bells and whistles like storing complete threads in single HTML files and updating those files as the threads get more posts.Quince - 17-2-2005 at 15:29
I think 7z uses better compression than gz, and the fact that few have the decompressor is moot given that a self-extracting archive can be made.Polverone - 17-2-2005 at 15:43
Wil it self-extract no matter what CPU and OS the user has? No, I didn't think so. There are certainly more powerful compression schemes than
gzip available, but few are more widely supported. I'm due to upload another encrypted backup of the forum just as soon as axehandle's FTP
server starts working again.Eliteforum - 19-2-2005 at 05:34
Polverone, I can host it on my HTTP if you want? PM me and I'll give you the details.PainKilla - 24-2-2005 at 19:50
My computer is always on due to various things running, and i have a few good progs for downloading sites....
If I have your permission Polverone I can download and safely store such a backup... you can do whatever you like with it. I can zip it up and send it
to you if you want to encrypt it.
I dont see the point of encryption though, except to protect members though i think we are as much at rish now than ever....neutrino - 24-2-2005 at 20:36
The reason for encryption (last I heard, anyway) was that the backups contain personal data by default and it would be impractical to remove it all.Polverone - 24-2-2005 at 21:59
That is correct. I have come up with a way to allow "open" archives, though. I would install mysql, php, apache, and the forum software on
my home machine. I would set up the forum like it's set up on the public server, and download/import database dumps from the public site. Then I
can set up a spidering program to grab all the public threads and finally make a zipfile archive with all these public threads.
Strictly speaking, it wouldn't be necessary to do this on my local machine, but it would take a considerable amount of time and bandwidth to do
it over the internet, using the live board. As it is, we are riding perilously near the edge of the bandwidth limit. I get 10 GB a month without
paying more. We'll have 11 GB of traffic this month if current trends continue.
Edit: it looks like about 20% of our traffic comes from the library, so I should be able to save bandwidth by shifting more books over to
BromicAcid's webspace. Thanks, BromicAcid!
[Edited on 2-25-2005 by Polverone]
new backup
Polverone - 12-3-2005 at 12:17
I have provided a fresh backup. It contains an encrypted copy of the board database, plus an unencrypted copy of all scipics files. You can download
it by connecting via FTP to 80.3.211.50 with username mad, password sci. Thanks go to eliteforum for providing the hosting for this bulky file.
Remember, if something catastrophic were to happen to me or our server, it might be your backup copy that saves the forum!Mahlzahn - 25-4-2005 at 06:33
I can`t connect to the ftp.
I have a timeout after 28 seconds.
