Interesting Integer Overflow

Fencliff · Aug 20th 2008

Hi All,

Found an interesting integer overflow condition today, thought I would share in case anybody else runs into same issue.

The condition arises when you use a literal int as the test expression in a Select Case statement:

Code

Sub ProducesSelectCaseOverFlowWhenTooManyRowsSelected()
    
    Select Case 2
    
        Case Selection.Rows.Count
            'Do stuff here
            
        Case Selection.Columns.Count
            'Do stuff here
            
        Case Selection.Areas.Count
            'Do stuff here
            
    End Select


End Sub

Display More

When the Selection.Rows.Count is larger than maximum 16-bit integer value 32,767, the "Case Selection.Rows.Count" line throws an overflow error.

I don't know how the compilation of Select Case constructs to p-code is implemented, but obviously each statement is evaluated in the (assumed) data type of the test expression.

The fix is easy: use a 32-bit integer (Long) constant as the test expression

Code

Sub DoesNotOverFlow()
    
    Const lVALID_REGION_COUNT As Long = 2
    
    Select Case lVALID_REGION_COUNT
        
        Case Selection.Rows.Count
        '...

**Andy Pope** · Aug 20th 2008

Re: Interesting Integer Overflow

Alternatives would be

[vba]
Select Case CLng(2)
[/vba]

[vba]
Select Case 2&
[/vba]

or store the value in an appropriately typed variable.

Interesting potentional error though.

**pike** · Aug 25th 2008

Re: Interesting Integer Overflow

Hi shg
why does this code need to have the CDbl() around the variable to stop an error when I have all ready stored the variable as a double ??

Code

Sub extraStuff(c As String)
Dim One As Double, Two As Double, Three As Double, MyControl As Control
Dim conOne As String, conTwo As String, conThree As String, conFour As String
conOne = "Salary" & c
conTwo = "Workforce" & c
conThree = "Hours" & c
conFour = "Price" & c
One = enigmaStuff(conOne)
Two = enigmaStuff(conTwo)
Three = enigmaStuff(conThree)
  For Each MyControl In Controls
    If (MyControl.Name Like conFour) Then
      MyControl.Value = CDbl(One) * CDbl(Two) * CDbl(Three)' overflow error on this line with large numbers
    End If
  Next
End Sub
Private Sub Workforce4_Exit(ByVal Cancel As MSForms.ReturnBoolean)
extraStuff (4)' textbox
End Sub
Private Sub Workforce5_Exit(ByVal Cancel As MSForms.ReturnBoolean)
extraStuff (5)' textbox
End Sub


Private Sub Hours4_Exit(ByVal Cancel As MSForms.ReturnBoolean)
extraStuff (4)' textbox
End Sub
Private Sub Hours5_Exit(ByVal Cancel As MSForms.ReturnBoolean)
extraStuff (5)' textbox
End Sub


Private Sub Salary4_Exit(ByVal Cancel As MSForms.ReturnBoolean)
extraStuff (4)' textbox
End Sub
Private Sub Salary5_Exit(ByVal Cancel As MSForms.ReturnBoolean)
extraStuff (5)' textbox
End Sub


Function enigmaStuff(b As String) As Double
Dim MyControl As Control
 For Each MyControl In Controls
   If (MyControl.Name Like b) Then
    Debug.Print MyControl.Value
    If MyControl.Value = "" Then
       enigmaStuff = 1
      Else
       enigmaStuff = CDbl(MyControl.Value)
     End If
   End If
 Next
End Function

Display More

or what coding error have i made?
pike

cytop · Sep 1st 2010

Re: Interesting Integer Overflow

This might be a little late - but the problem is related to the way VBA and (VB, plus I believe .NET as well) handles numeric variables.

For example:

Code

Public Sub Test
[INDENT]Dim x as Integer
Dim y as Integer
 
x = 200
y = 300
 
MsgBox (x*y)   <- Throws an error
[/INDENT]end sub

The reason for the error is that VBA creates a 'shadow' variable with a precision equal to the highest declared variable. In this case, Integer. It then uses that variable to store the results of the calculation before passing it back to the call to Msgbox. However, the variable is an integer so it overflows.

Even if you declare 'Z as Long' and try to assign Z = (X*Y) to it, the error will occur as VBA still creates an Integer shadow.

The only way around this is to declare one of the input variables as Long, then the shadow will be a long.

The same is happening here. AS Fencliff pointed out, the Case statements are assumed to have the same type as the Case comparison value (And 2 is an Integer) so it overflows when rows get to be greater than 32767 (or whatever...)

This problem has existed in Visual Basic since version 1. 0 and for all I know in the DOS QuickBasic as well...

J

Interesting Integer Overflow

Participate now!

Share