E-Mail:
Author Avatar

Web Spidering For You And Me: How To Find Those Pages That Others Can’t Or Won’t Find

This java application that I am going to start using is called ItSucks and is located here. I am going to be testing this little java app against my custom written in “C” spiders on my Linux machine. I will be using these spiders to find specialized Web pages that I have a hard time finding using the standard search engines like Google, Google Scholar, and Yahoo! I often search through university on line archives to find special articles that are often overlooked or orphaned. These articles are often missed because there are not a lot of connections to them, But the information included in these on-line papers can be invaluable in my work.

The results are in and I guess I am just not cut out to use complex regular expressions that are required to be used by this because I could not get the same results from my Linux C++ custom coded spiders, and this application.  So I am going to leave this app alone for right now until I can figure out how to the regular expression.  This does not mean that program does not work, it only means that I have some kind of mental block in using this application.

Tags: , ,

What Do You Think?

 


Anti-Spam Image

Want to Start a Blog Here for Free?

Are you an expert in one subject or another? If your goal is to help others and dispense hard-earned information back to the community, stake a claim on your very own Lockergnome blog today! You can write about anything - no matter the topic. Sign-up to start blogging!

54 queries / 0.426 seconds.