GetElementByID & Error checking

M40wen · Oct 3rd 2015

Hi all,

Just a quickie for advice. I am screen scraping but wanted to know the "best practice" for error checking the classname/tagname/elementid? I want my code to be as water tight as possible so was thinking a combination of checking the website title page for "Error" and also checking each and every element such as

was thinking of along the lines of

Code

Dim Srch as string


Srch = "ERRORHERE"
on error resume next
Srch = Trim(Doc.getElementsByClassName("col-a")(0).getElementsByTagName("ul")(0).getElementsByTagName("li")(i).getElementsByTagName("a")(0).getAttribute("href"))
on error goto 0
If srch = "ERRORHERE" then
' there is a problemo, go to end of loop and start on next line and record in comments of the element causing the issue
else
' found something

Display More

Is this the best way to handle it? Given website scraping throws up unexpected errors I want to trap, report and move on to next row as potentially the code could be looping hundreds of times, and fetching dozens of web elements so I cant afford for the macro to fall over or error ignored. Thought id ask here

cytop · Oct 3rd 2015

Re: GetElementByID & Error checking

As long as you code for it, 'On Error Resume Next' is as good a strategy as any other error handling.

The ~~simplest~~ thing to do is to check for errors after every interaction the website, there's no need to pre-seed a variable with a value and check if the variable still contains that value - you can simply check if Err.Number contains anything other than 0

Code

Sub x()
   
   Dim srch As String
   
   On Error Resume Next
   
   Do Some Loop
      
      srch = Trim(Doc.getElementsByClassName("col-a")(0).getElementsByTagName("ul")(0).getElementsByTagName("li")(i).getElementsByTagName("a")(0).getAttribute("href"))


      If Err.Number = 0 Then
         '// Do some more processing
         
         If Err.Number = 0 Then
             '// Do Yet more processing
         End If
      End If
      
      Err.Clear
      
   Loop


End Sub

Display More

Also, don't forget to clear any errors before going around the loop again.

The only drawback if you might end up with some very deep nested IFs - make sure you indent the code correctly so you can match End Ifs. If you don't have a tool for this then Smart Indenter is free, safe and I've never had a problem with it in x years. Don't be put off by the page mentioning "available free for Office 2000/2002/2003, Visual Basic 6 and Excel 97" - it works in Office 2013 (32 bit only, though).

M40wen · Oct 4th 2015

Re: GetElementByID & Error checking

Ok, i will go with this i just didnt want to start on the wrong path! I try and learn the best way and stick to that. As a side, wouldnt i be best off doing one if statement at a time rather than using a nested if? It would then check out for every error against the data item, of course if one errors they are all likely to error, but would keep the code simpler and still capture all errors? I would then run a second macro to filter on all the comments and process them for manual review. as long as i reset the code to a point, it should allow me to get as much info as possible

Many thanks for your help

cytop · Oct 4th 2015

Re: GetElementByID & Error checking

How you structure it is down to you, entirely.

Not too sure what you mean by "doing one if statement at a time rather than using a nested if", possibly to close the IF...ENDIF before trying the next step. There's no real difference but it might be useful to use an older language construct to control the program flow if you do that... and never mind anyone who sniggers at you for using GOTO - they use it every time they add an 'On Error...' statement.

Code

Do Some Loop 


        srch = Trim(Doc.getElementsByClassName("col-a")(0).getElementsByTagName("ul")(0).getElementsByTagName("li")(i).getElementsByTagName("a")(0).getAttribute("href")) 
         
        If Err.Number <> 0 Then Goto SomeLabelAtTheEndOfTheLoop
        
        '// Do some more processing
        If Err.Number <> 0 Then Goto SomeLabelAtTheEndOfTheLoop
                
SomeLabelAtTheEndOfTheLoop:         
        Err.Clear 
    Loop

Display More

The other alternative is to use the 'normal' error handling.

Code

On Error Goto ErrHandler


    Do Some Loop 


        srch = Trim(Doc.getElementsByClassName("col-a")(0).getElementsByTagName("ul")(0).getElementsByTagName("li")(i).getElementsByTagName("a")(0).getAttribute("href")) 
         
         '// Do some more processing
         '// More processing
                
SomeLabelAtTheEndOfTheLoop:         


    Loop
    
    Exit Sub


ErrHandler:


    '// Do Whatever
    Resume SomeLabelAtTheEndOfTheLoop


End Sub

Display More

But as the majority of the errors will probably be the very uninformation 'Error 1004' it'll be impossible to determine where the error occurred (unless you use that other old function ERL() and add line numbers to the code...

Like I said - down to you and whatever works best for you in the circumstances, just code for it.

M40wen · Nov 20th 2015

Re: GetElementByID & Error checking

Hi Cytop,

Thanks for the reply. I chose the following approach, but I have a slight issue with blank data (ie the field is found, but has no data in it). Basically all I want to know is if the field is not found, or if the date found is not a valid date (as I am trimming the field, it is possible that text will appear so I want this flagged for investigation). If it is found but purely holds no data, it should skip, but the line If Trim(Application.Clean(.getElementById("MainContent_tdApplicationRegistered").Value)) <> "" Then seems to make no odds whether it is null/nothing. Im guessing it doesn't make sense to use that and I should be extending the select case instead?

Code

ThisWorkbook.Sheets("Workbench").Cells(RW, wApplicationRegistered) = Format(DateValue(Trim(Application.Clean(Replace(.getElementById("MainContent_tdApplicationRegistered").innerText, "Application Registered ", "")))), "MM/DD/YYYY")
If Err.Number <> 0 Then
    Select Case Err.Number
        Case Is = 424
        ThisWorkbook.Sheets("Workbench").Cells(RW, wComments) = ThisWorkbook.Sheets("Workbench").Cells(RW, wComments) & "Error:Website Not found in " & Cells(RW, wApplicationRegistered).Address
        Case Else
            If Trim(Application.Clean(.getElementById("MainContent_tdApplicationRegistered").Value)) <> "" Then
            ThisWorkbook.Sheets("Workbench").Cells(RW, wApplicationRegistered) = Trim(Application.Clean(.getElementById("MainContent_tdApplicationRegistered").Value))
            ThisWorkbook.Sheets("Workbench").Cells(RW, wComments) = ThisWorkbook.Sheets("Workbench").Cells(RW, wComments) & "Error:check in " & Cells(RW, wApplicationRegistered).Address
            End If
     End Select
Err.Clear
End If

Display More

cytop · Nov 20th 2015

Re: GetElementByID & Error checking

I don't know what 'Application.Clean' returns if the input is blank - it could be a zero length string or a Null (Guess I could even find out), but you could try checking for the length of the returned string...

Code

If Len(Trim(Application.Clean(.getElementById("MainContent_tdApplicationRegistered").Value))) > 0  Then

M40wen · Nov 20th 2015

Re: GetElementByID & Error checking

It is saying it is empty...

Scrub that. The problem is the website is switching between .value and .innertext!!!! Sorry, I am sure I will be back to ask for more advice, but it is taking shape. This will end up a few thousand lines of code. This problem took me 2 hours!!!

M40wen · Nov 20th 2015

Re: GetElementByID & Error checking

Also it doesn't actually default a blank for that field, it looks like it will always default a question mark. Amended to this - do you think that is a good enough approach? There are approx. 60 fields a record I have to do this for

Code

ThisWorkbook.Sheets("Workbench").Cells(RW, wCommentsWelcomeBy) = Format(DateValue(Trim(Application.Clean(Replace(.getElementById("MainContent_tdCommentsWelcomeBy").innerText, "Comments Welcome By ", "")))), "MM/DD/YYYY")
If Err.Number <> 0 Then
    Select Case Err.Number
        Case Is = 424
        ThisWorkbook.Sheets("Workbench").Cells(RW, wComments) = ThisWorkbook.Sheets("Workbench").Cells(RW, wComments) & "Error:Website Not found in " & Cells(RW, wCommentsWelcomeBy).Address
        Case Else
            If Trim(Application.Clean(Replace(.getElementById("MainContent_tdCommentsWelcomeBy").innerText, "Comments Welcome By ", ""))) <> "?" Then
            ThisWorkbook.Sheets("Workbench").Cells(RW, wCommentsWelcomeBy) = Trim(Application.Clean(Replace(.getElementById("MainContent_tdCommentsWelcomeBy").innerText, "Comments Welcome By ", "")))
            ThisWorkbook.Sheets("Workbench").Cells(RW, wComments) = ThisWorkbook.Sheets("Workbench").Cells(RW, wComments) & "Error:check in " & Cells(RW, wCommentsWelcomeBy).Address & "/" & Err.Number
            End If
     End Select
Err.Clear
End If

Display More

cytop · Nov 20th 2015

Re: GetElementByID & Error checking

Quote

it looks like it will always default a question mark...

If it does, then fine, but I honestly don't know. Afraid this will be a 'suck it and see approach'.

GetElementByID & Error checking

Participate now!

Share