CEOP show Maddie is missing on 30th April 2007

Hongkong Phooey wrote:

WLBTS wrote:

Hongkong Phooey wrote:
Based on the same as everyone on here's actual knowledge of DBM which is very little! However the McCann file has the timestamp embedded into the URL and saying somehow this was allocated wrongly at a later date is just guesswork as well

The articles from the future also have that timestamp embedded into their URLs, and in their case the timestamp is completely wrong.

Can you demonstrate this?

It has already been demonstrated.  Look back at the articles also archived apparently on 30th April 2007, but which include much later news, and a later date as part of their filename.  However, they exist in the 20070430115803 folder, so that is also part of their URL.  Which is not correct.

If we're talking about filenames only, then the filename of that mccann page is 'mccann.html'.  I see no time-stamp there.

Mimi wrote:

Hongkong Phooey wrote:

Mimi wrote:At about 2:00 mins the founder of Waybackmachine says they respond to people wanting to take stuff off.

There is a link in the FAQs which you can ask to have your pages removed IIRC.

Right HKP, thanks. Sorry to be so thick.

No need to apologise I was also trying to be helpful

WLBTS wrote:

Hongkong Phooey wrote:

WLBTS wrote:

Hongkong Phooey wrote:
Based on the same as everyone on here's actual knowledge of DBM which is very little! However the McCann file has the timestamp embedded into the URL and saying somehow this was allocated wrongly at a later date is just guesswork as well

The articles from the future also have that timestamp embedded into their URLs, and in their case the timestamp is completely wrong.

Can you demonstrate this?

It has already been demonstrated.  Look back at the articles also archived apparently on 30th April 2007, but which include much later news, and a later date as part of their filename.  However, they exist in the 20070430115803 folder, so that is also part of their URL.  Which is not correct.

If we're talking about filenames only, then the filename of that mccann page is 'mccann.html'.  I see no time-stamp there.

They have the correct time stamp but are in the 30/04 folder no?

Hongkong Phooey wrote:
They have the correct time stamp but are in the 30/04 folder no?

No, they don't have the correct time stamp, otherwise they would not be in the 30th April time-stamp folder.  You're getting confused between time-stamps and the filenames of the articles which include their date.  The mccann page does not have a date in its filename at all, so what do you suggest is its time-stamp?

Brewster Kahle is on twitter. Couldn`t someone tweet him about this?

https://twitter.com/brewster_kahle

Nothing like going straight to the top.

froggy wrote:

Mimi wrote:Details of how it operated in September 2006,

http://www.enterprisestorageforum.com/technology/features/article.php/3633256/The-Wayback-Machine-From-Petabytes-to-PetaBoxes.htm

I suppose its not possible to access these back-up archives and examine the original, unaltered data ?

I too computer illiterate to know froggy. I just thought it was interesting info about the technical workings of Waybackmachine in 2006 and that it might be a help to the teccies here.

But it is interesting what someone has put in the comments section about the more the storage, the more the errors.

"By Kebabbert December 05 2009 18:30 PST
This *** badly. Dont you know that the more bits, the more bit rot you get? Large scale storage solutions MUST protect against bit rot, otherwise your data will silently flip bits. For instance, Lustre is being recoded to use ZFS right now. And one installation ZFS showed errors, the cause? A network card that silently injected random bits! If you have spent lots of money on a LHC collider, and you do not protect against bit rot, you are screwed. ZFS detected this quick. Earlier, they didnt even know of that faulty NIC. That *** badly. And the more storage, the more errors. ZFS head architect explains in his words why ZFS is the best on the market: http://queue.acm.org/detail.cfm?id=1317400 If you dont agree, ZFS is not for you. Your data is at risk. Read the short artcile, it may change how you view your data storage."

WLBTS wrote:

Hongkong Phooey wrote:
They have the correct time stamp but are in the 30/04 folder no?

No, they don't have the correct time stamp, otherwise they would not be in the 30th April time-stamp folder.  You're getting confused between time-stamps and the filenames of the articles which include their date.  The mccann page does not have a date in its filename at all, so what do you suggest is its time-stamp?

You got me there and I'm not knowledgeable enough to give a smart arse answer either

WLBTS wrote:

Hongkong Phooey wrote:
They have the correct time stamp but are in the 30/04 folder no?

No, they don't have the correct time stamp, otherwise they would not be in the 30th April time-stamp folder.  You're getting confused between time-stamps and the filenames of the articles which include their date.  The mccann page does not have a date in its filename at all, so what do you suggest is its time-stamp?

Do we at least agree that the Wayback creates or replays (as they call) it on retrieval? Can it be possible that because the 30/04 folder (at the exact time) has data from later dates that it has created something different? How do we know that there wasn't a part of the replayed file created on 30/04 and it was McCann.htm (hope I'm not driving you bonkers

Resistor your like a dog with a bone, thank god keep digging your doing a wonderful job for Madeleine she will be proud of you.

Hongkong Phooey wrote:
Do we at least agree that the Wayback creates or replays (as they call) it on retrieval? Can it be possible that because the 30/04 folder (at the exact time) has data from later dates that it has created something different? How do we know that there wasn't a part of the replayed file created on 30/04 and it was McCann.htm (hope I'm not driving you bonkers

Bonkers - not at all, pleased to have this conversation HKP

On replays, yes I'd expect it does that.  Systems are very rarely as simple as a read-write to a file-system.  What we view as 'folders' in a URL may be nothing of the kind - it's up to the web-server how it interprets a URL.  Sometimes parts of the URL are interpreted as commands.  It's web-server dependent.  And to the user, it shouldn't really matter what the implementation is - it should be a black box.  Here, a 'folder' is the abstraction of a folder, not an actual folder.  And of course, a folder on your PC or Mac is not a folder either, but I won't bore you with the technical details.

To put things simply:

The mccann.html page could be exactly what people originally thought it was - a page set up on 30th April 2007, several days in advance of the public alert to the disappearance of Madeleine McCann.

However, the evidence relating to it is now very poor indeed, as in the very same archive there are lots of pages that could not have been in existence on 30th April 2007.  This creates serious doubts about the mccann.html page, serious questions about whether it was indeed crawled on 30th April 2007.

My opinion based on the current evidence is that the entire 20070430115803 archive is very likely to have been incorrectly time-stamped.  Otherwise you have the incredible coincidence of CEOP allegedly putting up a page about Madeleine's disappearance several days before the public became aware of it, which would be an incredibly stupid act as Pat Brown says, and is a massive smoking gun that should blow the whole McCann case apart.  Couple that with an error that occurred possibly around December 2007 that put a load of pages that didn't exist in April into that very same time-stamp 'folder', bringing every page in that particular archive into question.  It didn't put them into June, or March, or 2006 - it put them into the exact time-stamp folder containing the smoking gun.  What are the odds of that?

WLBTS wrote:

Hongkong Phooey wrote:
Do we at least agree that the Wayback creates or replays (as they call) it on retrieval? Can it be possible that because the 30/04 folder (at the exact time) has data from later dates that it has created something different? How do we know that there wasn't a part of the replayed file created on 30/04 and it was McCann.htm (hope I'm not driving you bonkers

Bonkers - not at all, pleased to have this conversation HKP

On replays, yes I'd expect it does that.  Systems are very rarely as simple as a read-write to a file-system.  What we view as 'folders' in a URL may be nothing of the kind - it's up to the web-server how it interprets a URL.  Sometimes parts of the URL are interpreted as commands.  It's web-server dependent.  And to the user, it shouldn't really matter what the implementation is - it should be a black box.  Here, a 'folder' is the abstraction of a folder, not an actual folder.  And of course, a folder on your PC or Mac is not a folder either, but I won't bore you with the technical details.

To put things simply:

The mccann.html page could be exactly what people originally thought it was - a page set up on 30th April 2007, several days in advance of the public alert to the disappearance of Madeleine McCann.

However, the evidence relating to it is now very poor indeed, as in the very same archive there are lots of pages that could not have been in existence on 30th April 2007.  This creates serious doubts about the mccann.html page, serious questions about whether it was indeed crawled on 30th April 2007.

My opinion based on the current evidence is that the entire 20070430115803 archive is very likely to have been incorrectly time-stamped.  Otherwise you have the incredible coincidence of CEOP allegedly putting up a page about Madeleine's disappearance several days before the public became aware of it, which would be an incredibly stupid act as Pat Brown says, and is a massive smoking gun that should blow the whole McCann case apart.  Couple that with an error that occurred possibly around December 2007 that put a load of pages that didn't exist in April into that very same time-stamp 'folder', bringing every page in that particular archive into question.  It didn't put them into June, or March, or 2006 - it put them into the exact time-stamp folder containing the smoking gun.  What are the odds of that?

This is what is 'getting' people, if it's a genuine error then it's incredible that it picked that particular time/folder to put them in, almost unbelievable.

Hongkong Phooey wrote:
This is what is 'getting' people, if it's a genuine error then it's incredible that it picked that particular time/folder to put them in, almost unbelievable.

And with that you've hit the nail on the head.  Yes, it is unbelievable. That's what I've been saying since Wednesday.

More likely is that the entire time-stamp folder was crawled at some point in late 2007, and incorrectly stored as 20070430115803.

WLBTS wrote:

Hongkong Phooey wrote:
This is what is 'getting' people, if it's a genuine error then it's incredible that it picked that particular time/folder to put them in, almost unbelievable.

And with that you've hit the nail on the head.  Yes, it is unbelievable.  That's what I've been saying since Wednesday.

More likely is that the entire time-stamp folder was crawled at some point in late 2007, and incorrectly stored as 20070430115803.

With that sir I bid you goodnight and thank you for your patience, I may still not be convinced with your 'argument' but respect your viewpoint and knowledge.

Poppy wrote:Resistor your like a dog with a bone, thank god keep digging your doing a wonderful job for Madeleine she will be proud of you.

Yes! Where are you Resistor?...sitting back waiting patiently for your next post

Sigh ... Tony, seeing as you enjoy copying and pasting my words into your forum as you see fit, perhaps you could let Mr Hall know this:

You can look up the 'archive date' of all of those articles one by one, it doesn't demonstrate anything *now*.

Look up the original mccann.html page *now*.  Does it show an archive date of 30th April 2007 any more?  That's right, no it doesn't.

The data for the 20070430115803 time-stamp has been removed.  The mccann page redirects to another one now.  The home page redirects to another one now.  And *those articles* redirect now.

Go ahead, look up the archive date of the mccann page now.  According to your current logic, the mccann page was not archived on 30th April 2007 at all.  So case closed then!

I'm off to bed before I bang my head against a wall.

ETA - here you go Mr Hall, this is what the mccann page now shows:

Code:


<!--
     FILE ARCHIVED ON 2:09:01 May 13, 2007 AND RETRIEVED FROM THE
     INTERNET ARCHIVE ON 23:06:25 Jun 21, 2015.
     JAVASCRIPT APPENDED BY WAYBACK MACHINE, COPYRIGHT INTERNET ARCHIVE.

     ALL OTHER CONTENT MAY ALSO BE PROTECTED BY COPYRIGHT (17 U.S.C.
     SECTION 108(a)(3)).
-->


So, case closed, the mccann page was archived on May 13th 2007! That's what your current logic demonstrates.  Tell me why I'm wrong here, and that will tell you why you are wrong about the articles.

WLBTS wrote:Sigh ... Tony, seeing as you enjoy copying and pasting my words into your forum as you see fit, perhaps you could let Mr Hall know this:

You can look up the 'archive date' of all of those articles one by one, it doesn't demonstrate anything *now*.

Look up the original mccann.html page *now*.  Does it show an archive date of 30th April 2007 any more?  That's right, no it doesn't.

The data for the 20070430115803 time-stamp has been removed.  The mccann page redirects to another one now.  The home page redirects to another one now.  And *those articles* redirect now.

Go ahead, look up the archive date of the mccann page now.  According to your current logic, the mccann page was not archived on 30th April 2007 at all.  So case closed then!

I'm off to bed before I bang my head against a wall.

ETA - here you go Mr Hall, this is what the mccann page now shows:

Code:


<!--
     FILE ARCHIVED ON 2:09:01 May 13, 2007 AND RETRIEVED FROM THE
     INTERNET ARCHIVE ON 23:06:25 Jun 21, 2015.
     JAVASCRIPT APPENDED BY WAYBACK MACHINE, COPYRIGHT INTERNET ARCHIVE.

     ALL OTHER CONTENT MAY ALSO BE PROTECTED BY COPYRIGHT (17 U.S.C.
     SECTION 108(a)(3)).
-->


So, case closed, the mccann page was archived on May 13th 2007!  That's what your current logic demonstrates.  Tell me why I'm wrong here, and that will tell you why you are wrong about the articles.

I saw that as well, no idea where he's going. Another quick question. If the WBM reconstructs pages rather than snapshots them then will it reconstruct within the folder (I would have thought so). If it doesn't find the exact data it uses the closest to it (so it says in the FAQs) so.... Is the McCann.htm the master in 20070430115803 and WBM has constructed around the files in the folder? Does that make any sense?

Err... I have just come home!  Believe it or not, I do have a life outside this forum!

I really wish I could waltz in and give you all a sparkling explanation that explains it all to everyone's satisfaction, but I can't, sorry  I have zero explanation for stuff being dated October in a file, ostensibly created in April, but if it's not as a result of some dynamic content, then yes.  It is an anomaly and has to be explained.

We know that something was created on 30 April 2007 when the CEOP site was crawled, because pages were saved and an index (folder) created for it.  I have no reason to doubt that date at all.  30 April 2007 at 11:58:03.  For reasons I have already given (about 30 pages back now :0 ) I trust server timers unless there is a very good reason for them to be "wrong".  And when they do go wrong, they go spectacularly wrong.  It would be saying something like 1 January 1900 if it was really, really wrong.  

So now the school of thought seems to be is that the site was again crawled sometime after late October 2007.  In actual fact the first date after 27/10/07 is 6/2/08.  Their Javascript that automatically appends confirms this.  That seems an awfully long gap, looking at the other captures in the calendar, so is there a missing capture in the period from 12/10/07 to 6/2/08?  If so, that capture wasn't saved in it's own folder during that period, it was diverted - by presumably some sort of software pointer error - to the April folder, where it went in with the April files that were already there.

(ETA - why April 30, and not Oct 12, the nearest one to it?)

If you try, in Windows, to move a file into a folder, and there is already a file there of the same name, it asks you what you want to do.  If you choose to save the newer one, it overwrites the old one.  I don't know what sort of server Wayback sits on, but I do have some experience of UNIX and APACHE and if you try to replace a file, it just overwrites it, with no warnings or dialogs.  I can well imagine that the October-to-February version was saved to the April index and just overwrote the April version, but why it would do that in the first place, I have no idea.

So this now leaves us with when mccann.html was actually created and crawled.  Was it there in April and went in the folder originally?  Or was it only there in October-to-February and went in with the April stuff as the same time as the later homepage?

The bit that is gnawing at me is the lack of the second photograph. Madeleine_02.jpg, in the April version.  It should have been there on the April page, but it wasn't, because the webpage showed a broken link.  So if it was an October-to-February page was saved into the April index, the whole thing was not saved, because then we would not be minus a photo.  Later versions have both photos.  So clearly the saving process was not exactly the same in all cases.

Now a couple of nights ago, HKP very helpfully found some stuff in Wayback's own FAQ that tells us how the pages are replayed when they have only saved part of them.  They try to reconstruct it as best they can from the next nearest version.  For mccann.html that would have been May 13, which has both photos, but not the little flags that appear in even later versions.  The May version tells us in the appended Javascript that the previous capture was April 30.



If there wasn't a version of mccann.html in the folder on 30 April (as it was only added at some later point October-February) then how did a file created on 13 May manage to find it, to add it in as a previous version?  Because the 13 May capture should have been the first one.

Sorry, it's late now and I am probably not expressing myself very clearly.  I also need to be up early in the morning.  I'll give this some more thought over the next couple of days, and email Wayback again, as so far they have not responded to any of my queries.  I suspect it's the same for us all.

This is along the lines of what I was thinking however I don't have enough knowledge to get there, my view is that there was a McCann file on 30/04 and it is the reconstruction on retrieval which is finding later pages / links and adding them to the what I would loosely term master. I'm no techie though!

HKP. you have no idea how depressed this has all left me now. I still feel, really feel, that we have found something very significant indeed. And it is something I wish wasn't true, because it's evidence that something truly evil and horrific has happened, and a wee girl is dead. But I still remain convinced, because I know what I know and there's no way of unknowing it - and it is upsetting me very greatly.

And there is no way we can prove any of it, not without answers, sensible coherent definitive answers, from Wayback. ("A glitch" and "a mistake" are not definitive answers, BTW). We are never going to get those answers. It seems many people have asked them directly, all have been ignored, apart from Isabelle and Lizzie - and they only got answers because they asked before Wayback wised up to what was going on. And even then, the answers were in direct contradiction to one another, so it actually tells us nothing.

We won't get answers not because of some big CEOP or Government coverup, but because Wayback have to protect their reputation, or at least what's left of it. Because this has been such a hotly contested, divisive topic, in only a few days time, anyone who Googles something like "how accurate is Wayback" will be directed straight to one of these forums. Where the whole thing has been analysed to the nth degree and totally ripped apart by a lot of very knowledegable people. Wayback advertise themselves as an internet archive, but as the whole purpose of an archive is to preserve an accurate record, they are finished.

Nor do I think that Wayback were "leaned on". I think they might have been asked to remove the offending pages. They state in their own FAQ that they have no problem with that, and also that site owners can block their crawls with a robots file. I had a good look last night and I found quite a few, even pretty innocuous sites like the English FA.) So I'm not sure how this contributes to a "complete" Internet archive; that's a bit of a mixed message in their very own mission statements, right there.

So did mccann.html exist on 30 April 2007? Yes, I believe it did. Unfortunately, I cannot prove that it did. And neither can anyone else without a full explanation from Wayback, which we are not going to get.

Team mccann seem to have more lives than a bloody cat. What are the odds of this happening to this particular site?! I could just rip my own hair out with the sheer frustration and unfairness of it all. If I'm not seen for a couple of days, it's just that I am taking a break from this whole sorry mess, because I have been neglecting things that I should have been doing over the past couple of days and I have to get on with my own life.

If anyone ever does get any sort of response from Wayback, I'll be straight all over it like a bad rash, though.

Presumably the converse is also true, and no one can prove that it didn't exist - other than the Wayback people who have the original source material.

What I find difficult to understand is why Wayback people has changed the archive page details, rather than simply suspend them while a full investigation takes place. This, to me, suggests interference.

And here's another wee bit of food for thought. (I'm late for work now :0)

Richard D Hall's videos are very popular on the internet, they have received millions of hits. Now what if a popular indie filmmaker like RDH were to make a nice video all about Wayback and how accurate, or inaccurate, it is. I'm sure lots of people would be very keen to see it. And I'm also sure that Wayback would be forced to respond.

froggy wrote:

Presumably the converse is also true, and no one can prove that it didn't exist - other than the Wayback people who have the original source material.

What I find difficult to understand is why Wayback people has changed the archive page details, rather than simply suspend them while a full investigation takes place. This, to me, suggests interference.

Yes, why not leave it in place with some sort of disclaimer notice, that it was being investigated.  I also don't understand the indecent haste to erase it all, or the lack of responses from them since.

I really am leaving for work now before I miss my bus!!!

Why does WBM now point you to the 13th May for the archive when Chris Butler in his second email claims the original archive should read 31st July and it's all an error?

May I just say we owe a huge debt to Resistor and WLBTS and also the other people who have dedicated hours to this. HKP, Andyb etc. Also those who have unearthed useful info to help the cause.
Ok so the 2 main contributors have opposing views but no one can deny their motive is to prove this one way or the other.
Huge well done. 
Resistor, I understand your frustration. When one is certain of something one can become passionate, often to the detriment of normal life. I hope you take a wee break then come back refreshed.
Fwiw, I am of the same opinion as you without the impressive knowledge. It is just all too convenient to blame a glitch on this particular instance.
If something smells like bs it's usually bs.

Keep the faith Resistor you're doing some sterling work which we all hope will be worthwhile, same goes for WLBTS & Andyb