Java Web Scraper using JSoup – Part IV

In this tutorial we’ll be scraping a webpage with a set of list items. This tutorial is useful for beginners in web scraping. If you are expecting some advance stuff, I will be posting more of those tutorials soon but for now you can read through or just skip this part.

The address that we’ll be using here is http://bemorewithless.com/my-100-thing-challenge/

Now, we’ll be fetching the 100 items that are listed in the site.

Here’s our code.


package org.soup.examples;

import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;
import org.jsoup.select.Elements;

import java.io.*;

public class FectchList{

public static void main(String[] args) throws IOException{
String input = "http://bemorewithless.com/my-100-thing-challenge/";
Document doc = Jsoup.connect(input).get();

Elements items= doc.select("div.postarea ol li");

for(Element item:itemss){
System.out.println(item.text());
}
}
}

Run the program and the output should be the 100 items listed on the website. Now, we used the same library as before but we changed the url that the input variable contains.

You may be wondering why did I put div.postarea before ol and li. Well, its because the list is contained within the div tags with the class name of postarea. You can see those names if you view the page source.

Thank you for reading. Don’t forget to leave a comment and share.

Advertisements

9 thoughts on “Java Web Scraper using JSoup – Part IV

  1. I rarely drop remarks, but i did some searching and wound
    up here arjunaraneta | Java Web Scraper using JSoup – Part IV.
    And I actually do have 2 questions for you if you do not mind.
    Is it only me or does it give the impression like a few
    of the responses look like written by brain dead folks?

    😛 And, if you are posting at other places, I’d like to follow everything new you have to post. Would you make a list of every one of your shared sites like your linkedin profile, Facebook page or twitter feed?

  2. If some one needs expert view concerning blogging afterward i recommend
    him/her to pay a quick visit this website, Keep up the fastidious job.

  3. Thanks for the marvelous posting! I quite enjoyed reading it, you happen to be a great author.I will ensure that I bookmark your blog and
    will come back very soon. I want to encourage you continue your great work, have a nice day!

  4. Hello outstanding website! Does running a blog such as this take a great deal of work?
    I’ve very little expertise in computer programming but I had
    been hoping to start my own blog in the near future.
    Anyway, if you have any suggestions or techniques for new blog owners please
    share. I know this is off topic nevertheless I
    simply had to ask. Thanks a lot!

  5. This website was… how do you say it? Relevant!! Finally
    I have found something that helped me. Kudos!

  6. Hi! This is kind of off topic but I need some help from an
    established blog. Is it tough to set up your own blog?

    I’m not very techincal but I can figure things out pretty quick.
    I’m thinking about creating my own but I’m not sure where to
    start. Do you have any tips or suggestions? Many
    thanks

  7. Whoa! This blog looks just like my old one! It’s on a totally different topic but it has pretty much the same page layout
    and design. Superb choice of colors!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s