Website ripping

Discussion in 'Computer Science & Culture' started by s0meguy, Jul 3, 2008.

Thread Status:
Not open for further replies.
  1. s0meguy Worship me or suffer eternally Valued Senior Member

    Messages:
    1,635
    I have been trying several website ripping programs, but they all do it by following the links they find in the website. I need a program that just downloads every single file on the webserver, even if there is no link to it. Is this possible?
     
  2. Google AdSense Guest Advertisement



    to hide all adverts.
  3. Dr Mabuse Percipient Thaumaturgist Registered Senior Member

    Messages:
    714
    i've had very good results using Adobe Acrobat and making a PDF of a site in whole, you can also choose how many links deep you want to capture too...
     
  4. Google AdSense Guest Advertisement



    to hide all adverts.
  5. RubiksMaster Real eyes realize real lies Registered Senior Member

    Messages:
    1,646
    That would only work if you had the proper permissions on the server, which you won't, unless for some weird reason they set up their server wrong. At the most, you would only be able to download what's in the public directory, and without permission to view the filesystem, then all you could do was guess which files are on there.

    I could be wrong though. After all, google does it somehow. But then again, it can only crawl the public directory, it can't access every file on the server.
     
  6. Google AdSense Guest Advertisement



    to hide all adverts.
  7. firdroirich A friend of The Friends Registered Senior Member

    Messages:
    565
    Teleport pro could be what you're after, I've used on and off for years, couldn't find anything better for what I needed it to do - here is a list of features..

    - Download all or part of a website to your computer, enabling you to browse the site directly from your hard disk at much greater speeds than if you were to browse the site online
    - Create an exact duplicate, or mirror of a website, complete with subdirectory structure and all required files

    - Search a website for files of a certain type and size

    - Download a list of files at known addresses

    - Explore every website linked from a central website

    - Search a website for keywords

    - Make a list of all pages and files on a website
     
Thread Status:
Not open for further replies.

Share This Page