Web Scraping With VBA Using Selenium Reference Library

Important Notice

Please note that on 14th December 2023 users will experience an expected outage whilst we make upgrades to our network. We anticipate this process may take a couple of hours and so we apologise in advance for any inconvenience.

  • I have been working VBA to scrape website HTML code using the Microsoft HTML Object Library and the Microsoft Internet Controls Library. Now that support for Internet Explorer is being phased out, I am trying to switch my code over to scrape Google Chrome using the Selenium Type Library which is an open source download. The website I am trying to pull data from into excel has the following line of HTML code that I am interested in assigning to a variable:

    <div jsname="iXWWee" class="enWFYd KDN9Hf" style="left: 39px; display: block; transform: translate3d(85px, 0px, 0px);">Feb 4, 2021</div>

    I have been able to successfully assign a value to a variable using the following lines of code

    But what you will notice is that the the HTML code I provided has innerHTML which changes based on where your cursor is on the screen. The code above works but it only pulls the last instance where the innerHTML value is "Aug 3, 2021". What I want is something that will pull all of the data and not just the last instance. I tried to test it by changing the second to last line of code to Set Element = ch.FindElementByCss("div[class='enWFYd KDN9Hf'] [innerHTML='Feb 4, 2021']") but when I run the subroutine, I get an error saying that this element can't be found. I have also tried changing the last three lines as follows but it still is only pulling the last instance of this div tag.

    Dim Elements As Selenium.WebElements
    Set Elements = ch.FindElementsByCss("div[class='enWFYd KDN9Hf']")
    For Each Element in Elements
        Debug.Print Element.Attribute("innerHTML")
    Next Element

    Am I not setting up my Css selectors correctly? Anybody that is familiar with how to use Selenium with VBA, I would greatly appreciate your help.

  • Quote


    5a. No asking to crack passwords of any kind. Doing so, even innocently, will get you banned.

    5b. No asking for help with homework or college assignments. We will not do it for you. This also applies to the hire Help forum.

    5c. Web scraping. Ozgrid will not assist you in systematically acquiring data (even in the public domain) from a other peoples websites. We will show you how to do excel/browser automation with generic examples and answer specific questions related to browser automation VBA methods

    NOTE 5C

Participate now!

Don’t have an account yet? Register yourself now and be a part of our community!