VA 7.0 - Oracle Database Framework Bugs

VA Smalltalk is a "100% VisualAge compatible" IDE that includes the original VisualAge technology and the popular VA Assist and WidgetKit add-ons.

Moderators: Eric Clayberg, wembley, tc, Diane Engles, solveig

VA 7.0 - Oracle Database Framework Bugs

Postby cknittel » Tue Nov 28, 2006 7:03 am

Hi all,

I have taken the time to compile a list of bugs in the VisualAge 6.0.3/7.0 Oracle Database Framework that I have encountered over time. In the cases where I found a solution, I have included the fix.

It would be great if these issues could be addressed in the upcoming VA 7.5 release, especially if the database framework is going to be changed for Oracle 10g and LOB support anyway.

Here is a list of the problems. Detailed descriptions follow below.

1.) Sporadic error message "AbtError does not understand >"
2.) OciBreak thread is not terminated on disconnect
3.) Cannot invoke stored procedures without parameters
4.) AbtDatabaseConnection>>#unRegister should not set databaseMgr to nil
5.) Resource leak if connect fails
6.) NUMBER(x,0) is handled as a scaled decimal, not as an integer
7.) nullsOk is preset incorrectly for host variables
8.) implementsStoredProcs should be a class method
9.) Oracle RAW fields broken with 6.0.3 change
10.) Stored procedure parameter lists hold references to freed OS objects after disconnect
11.) Raising exceptions in an error block causes cursor leak

Thanks and best regards,
Christoph


1.) Sporadic error message "AbtError does not understand >"

Reason: AbtOracleDatabaseConnection>>#errorTextFor: does not check for AbtError

It can occur that #callPlatformFunction:withArray:useThreadPreference:threadKey: returns an AbtError.
This case is not handled properly.

Fix: Change AbtOracleDatabaseConnection>>#errorTextFor: as follows.

Code: Select all
errorTextFor: aReturnCode
   "Answer the error text for the specified return code."

   | textLength buf bufLength |

   self isTraceActive ifTrue: [ self logTraceMessage: AbtDbmTraceGetMsgText. ].

   buf := ByteArray new: Oci_Max_Error_Msg_Length.
   bufLength := buf size.
   textLength :=
      self
         callPlatformFunction: OracleOciOerhms
         withArray:
            (Array with: (self lda abtAsOraclePassedPointer) with: aReturnCode with: buf with: bufLength)
         useThreadPreference: true
         threadKey: nil.

   textLength isAbtError
      ifTrue: [
         ^'Internal error retrieving error text for %1: %2'
            bindWith: aReturnCode printString
            with: textLength errorText
      ].

   ^textLength > 0
      ifTrue: [ (buf memcpyStringFrom: 0 to: textLength - 1) abrAsString. ]
      ifFalse: [ 'Bogus error code'. ]



2.) OciBreak thread is not terminated on disconnect

Calling AbtOracleDatabaseConnection>>#breakIfError: causes an additional thread named '<ConnectionAlias>_OciBreak' to be created in #breakCallForLda:ifError:. This thread is not terminated on disconnect.

Fix: Add the following method in AbtOracleDatabaseConnection.

Code: Select all
terminateThread
   "Terminate thread for 'break' calls, too."

   super terminateThread.

   self alias notNil ifTrue: [ AbtThreadManager terminateThread: self alias , '_OciBreak'. ].



3.) Cannot invoke stored procedures without parameters

AbtOracleAbstractDatabaseConnection>>#invokeStoredProcedureNamed:withParameters:ifError: cannot handle stored procedures without parameters.

Fix: Change the method as follows.

Code: Select all
invokeStoredProcedureNamed: aProcName withParameters: aParmList ifError: anErrorBlock
   " "
   | statementStream |

   " Move this code into the spec itself "

   (statementStream := WriteStream on: (Locale current preferredStringClass new: 128))
      nextPutAll: 'BEGIN ';
      nextPutAll: aProcName.

   (aParmList isNil or: [ aParmList parameters isEmpty. ])
      ifTrue: [
         statementStream nextPutAll: '; END;'.
         ^self executeSQLStatement: statementStream contents ifError: anErrorBlock
      ]
      ifFalse: [
         statementStream nextPutAll: '( '.

         aParmList doWithIndex: [ :aParm :anIndex |
            aParm type fieldsDo: [ :aParmField |
               anIndex > 1 ifTrue: [ statementStream nextPut: $,. ].
               statementStream
                  nextPut: $:;
                  nextPutAll: aParmField name.
            ].
         ].

         statementStream nextPutAll: '); END;'.

         ^self
            executeStoredProcQuerySpec: statementStream contents abrAsQuerySpec
            withParameters: aParmList
            ifError: anErrorBlock
      ].



4.) AbtDatabaseConnection>>#unRegister should not set databaseMgr to nil

AbtDatabaseConnection>>#unRegister was changed in 6.0.2 (due to PQ77138, according to the comment):
- "critical" was added
- databaseMgr was set to nil

The latter caused a problem in our code, but I am not sure anymore where exactly. ;-) Possibly in #isTraceActive.

Fix: Leave the "critical" in there, but do not set databaseMgr to nil

Code: Select all
unRegister
   "Delete the receiver from the collection of active connections"

   [self databaseMgr removeActiveConnection: self] critical



5.) Resource leak if connect fails

AbtOracleDatabaseManager>>#connectToDataSourceNamed:logonSpec:ifError: leaks resources leak if the connect fails.

After a lot of retries I get the error message "couldn't allocate a static future".

Fix: Invoke "workQueue terminate" and "terminateThread"

Code: Select all
connectToDataSourceNamed: aDataSourceName logonSpec: aLogonSpec ifError: anErrorBlock
   " Establish a new database connection.  Answer the new AbtDatabaseConnection or
     an AbtError. "

   | rc newConnection |

   newConnection := self newDatabaseConnection dataSourceName: aDataSourceName.
   rc :=
      newConnection
         logonToDataSource: aLogonSpec server
         id: aLogonSpec id
         pw: aLogonSpec password
         ifError: anErrorBlock.

   ^rc isAbtError
      ifFalse: [ newConnection. ]
      ifTrue: [
         newConnection
            freeHda;
            freeLda.

         newConnection workQueue terminate.
         newConnection terminateThread.
         rc.
      ]



6.) NUMBER(x,0) is handled as a scaled decimal, not as an integer

(a) Rows returned by a single or multiple row query contain ScaledDecimal values for fields of type NUMBER(x,y), even if y=0. However, logically, fields of type NUMBER(x,0) should be treated as (Large)Integers.
(b) The same holds for the type of host variables that is preset after creation of a query.

Partial fix for (b): Change AbtOracleAbstractDatabaseManager>>#fieldForColData: as follows.

Code: Select all
fieldForColData: aColData
   "Fix: recognize integer host variables correctly."

   | fieldType precision scale |

   fieldType := self class typeStringDict at: aColData columnType.
   precision := aColData precision ifNil: [ 0. ].
   scale := aColData scale ifNil: [ 0. ].

   (fieldType = Oci_Number_Datatype and: [ scale = 0 and: [ precision ~= 0. ]. ])
      ifTrue: [ fieldType := Oci_Integer_Datatype. ].

   ^self class
      fieldForType: fieldType
      length: aColData length
      precision: precision
      scale: scale
      nullOk: aColData acceptsNulls



7.) nullsOk is preset incorrectly for host variables

#fieldForType:length:precision:scale:nullOk: has three implementors:
- AbtOdbcDatabaseManager
- AbtOracleDatabaseManager
- AbtOracle8DatabaseManager

The first expects the nullOk parameter to be a boolean, the latter two expect it to be an integer.

There are six senders of this method (Oracle + ODBC). Some of the Oracle ones pass a boolean, others an integer.

Fix: Standardize to boolean.


8.) implementsStoredProcs should be a class method

#implementsStoredProcs is sent by AbtStoredProcSettingsView to the database manager *class*.

However, in AbtDatabaseManager, it is implemented incorrectly as an instance method.

Fix: Make #implementsStoredProcs a class method in AbtDatabaseManager


9.) Oracle RAW fields broken with 6.0.3 change

In 6.0.3, AbtDatabaseBinaryField>>#put:intoRecord:parentOffset: was changed (obviously for some ODBC-related fix).

Unfortunately, this change broke the subclass AbtOracleRawField, because the invoked method #valueLengthFor: exists only for ODBC.

Fix/Workaround: Undo the change.


10.) Stored procedure parameter lists hold references to freed OS objects after disconnect

- An AbtStoredProc stores the list of parameter values for the stored procedure call in the instance variable "inputs".
- The parameter values are instances of AbtOracleRow and contain several OS pointers (columnLengths, returnCodes, ...).
- When the stored procedure is invoked, a PL/SQL block of the form "begin MyProc(myArguments); end;" is executed. The block also holds references to the parameter values, but not to the list.
- After the second invocation (?), the block is cached together with its arguments.
- On disconnect, the cache is flushed (flushCacheIfError:). This causes the OSPtr instances for the parameter values to be freed.
- The parameter list object in the stored procedure does not notice that.
- On the next stored procedure call (after a database reconnect), the broken parameter list with the freed pointers is passed and an exception is raised: "UndefinedObject does not understand abtAsSegmentedAddress"

Workaround: After database reconnect, set the instance variable "inputs" to nil.


11.) Raising exceptions in an error block causes cursor leak

As documented in Object class>>#abtDefaultDatabaseErrorBlock, returns from a database error block are not allowed.

I consider this a design bug. It should be possible to return/raise exceptions in an error block.

Fix: Use #ensure: in the framework methods to free resources (easier said than done, this would have to be changed in a lot of places ... ;-))
cknittel
 
Posts: 27
Joined: Tue Nov 28, 2006 6:10 am
Location: Vienna - Austria

Re: VA 7.0 - Oracle Database Framework Bugs

Postby marten » Wed Nov 29, 2006 9:49 am

:D - good posting ! Applause !
marten
[|]
 
Posts: 641
Joined: Sat Oct 14, 2006 7:10 am
Location: Hamburg - Germany

Postby solveig » Wed Nov 29, 2006 12:52 pm

c:

What a bundel! We will certainly evaluate them for inclusion in the next release.

Thanks,
Solveig
solveig
Moderator
 
Posts: 57
Joined: Tue Oct 17, 2006 6:30 am

Postby cknittel » Fri Dec 01, 2006 6:46 am

@Marten: Thanks! :P That motivates me to post a similar list about Windows GUI Framework issues... (it will probably be shorter)

@Solveig:

Great!

BTW, in my posting I forgot to mention that I am still using the "AbtOracle*" classes, not the newer "AbtOracle8*" ones.

The reason is that the "AbtOracle*" classes work fine for me (with the above modifications) for all Oracle versions from 7 to 10g, but I never got the "AbtOracle8*" classes to work properly for our application. As far as I can remember, LONG and/or LONG RAW fields were the main issue. If you like I can dig this up and post the details.

A general thought regarding VA bugs/bug fixes:

Maybe it is too easy in VA (and in Smalltalk in general) for developers to patch IDE and framework problems themselves, so that they don't bother reporting them to the vendor.
Therefore it might make sense to offer them some incentive to do so. Maybe a bug/fix of the week/month/year competition? ;-)

Regards,
Christoph
cknittel
 
Posts: 27
Joined: Tue Nov 28, 2006 6:10 am
Location: Vienna - Austria

Postby solveig » Mon Dec 04, 2006 9:04 am

Cristoph,

Monthly recognition of bug/fix contributions is a great Idea.

After the 7.5 release, we plan to add a beta release page to our web-site. We could recognize the community contributions to the latest beta there, including the "best." :wink:

Solveig
solveig
Moderator
 
Posts: 57
Joined: Tue Oct 17, 2006 6:30 am

Postby tc » Thu Dec 07, 2006 1:06 pm

Hello,

'As far as I can remember, LONG and/or LONG RAW fields were the main issue. If you like I can dig this up and post the details. '

. . . can you post that issue also and is it related to issue #9 in your list or a totally separate issue?

Thanks.

--tc
tc
Moderator
 
Posts: 304
Joined: Tue Oct 17, 2006 7:40 am
Location: Raleigh, NC

Postby tc » Thu Dec 07, 2006 8:39 pm

Hello,

I worked on the list you posted using Oracle 10 (logged on as Oracle 10). Here is what I found:


~~~~~~~~~~~~
Could Recreate
~~~~~~~~~~~~

I was able to reproduce the problem with these items:

3. Cannot invoke stored procedures without parameters.
7. In the method #fieldForType:length:precision:scale:nullOk:, 'nullsOk' is handled differently
8. ImplementsStoredProcs should be a class method.


~~~~~~~~~~~~~~~~~~~~~~~~~~~
Could Recreate but Need More Info
~~~~~~~~~~~~~~~~~~~~~~~~~~~

9. Oracle RAW fields broken with 6.0.3 change
I can see the error you describe but would like to be able to recreate it. I created a table with a 'long raw' column and saved pictures into it but was unable to get an error.


~~~~~~~~~~~~
Need more Info
~~~~~~~~~~~~

These are items I would like to get more information about how to create the error:

4. AbtDatabaseConnection>>#unRegister should not set databaseMgr to nil. The latter caused a problem in our code, possibly in #isTraceActive.

5. Resource leak if connect fails. After a lot of retries I get the error message "couldn't allocate a static future".
I tried connecting with a valid id/invalid pw and Oracle locked the account. I also tried with an invalid id/pw and each time, I got back an invalid logon message. I used a loop of 200 iterations.

6. NUMBER(x,0) is handled as a scaled decimal, not as an integer.
I created tables with smallint, int, and number, however, the way ST works, if the number is smaller than 2**31 - 1 then it comes back ok, larger than that and I get back a zero.

10. Stored procedure parameter lists hold references to freed OS objects after disconnect.
I connected and disconnected many times and called a stored proc but did not get that error.

--tc
tc
Moderator
 
Posts: 304
Joined: Tue Oct 17, 2006 7:40 am
Location: Raleigh, NC

Postby tc » Fri Dec 08, 2006 11:26 am

Hello,

For:

'9. Oracle RAW fields broken with 6.0.3 change'

. . . I was able to reproduce the issue and test the fix.

--tc
tc
Moderator
 
Posts: 304
Joined: Tue Oct 17, 2006 7:40 am
Location: Raleigh, NC

Postby cknittel » Mon Dec 11, 2006 7:49 am

Hi tc,

Thank you for the feedback! I will post the AbtOracle8* issues as soon as I find some time. :wink:

4. AbtDatabaseConnection>>#unRegister should not set databaseMgr to nil.

I get the following walkback on disconnect:

Code: Select all
=============<STACK TRACE BEGINS>============
[] in <optimized block>(ExceptionalEvent class)>>#initializeSystemExceptions
   signal=Signal on Exception: (ExError) An error has occurred.
Signal>>#evaluate:
   self=Signal on Exception: (ExError) An error has occurred.
   aBlock=[] in ExceptionalEvent class>>#initializeSystemExceptions
ExceptionalEvent>>#applyDefaultHandler:
   self=Exception: (ExError) An error has occurred.
   aSignal=Signal on Exception: (ExError) An error has occurred.
   exception=Exception: (ExError) An error has occurred.
ExceptionalEvent>>#signalWithArguments:
   self=Exception: (ExError) An error has occurred.
   arguments=('UndefinedObject does not understand isTraceActive')
   signal=Signal on Exception: (ExError) An error has occurred.
   handler=nil
ExceptionalEvent>>#signalWith:
   self=Exception: (ExError) An error has occurred.
   arg1='UndefinedObject does not understand isTraceActive'
UndefinedObject(Object)>>#error:
   self=nil
   aString='UndefinedObject does not understand isTraceActive'
UndefinedObject(Object)>>#doesNotUnderstand:
   self=nil
   aMessage=Message (#isTraceActive, ())
AbtOracleDatabaseConnection(AbtDatabaseConnection)>>#isTraceActive
   self=an AbtOracleDatabaseConnection
AbtOracleDatabaseConnection>>#logoffFromLda:ifError:
   self=an AbtOracleDatabaseConnection
   aLda=AbtRecord of type:
  'v2ReturnCode': 0
  'filler1': ''
  'returnCode': 0
  'filler2': ''
  'OSDErrorCode': 0
  'checkByte': $
  'filler3': ''
   anErrorBlock=[] in AbtDbmSystem class>>#abtDefaultDatabaseErrorBlock
AbtOracleDatabaseConnection>>#disconnectIfError:
   self=an AbtOracleDatabaseConnection
   anErrorBlock=[] in AbtDbmSystem class>>#abtDefaultDatabaseErrorBlock
   rc=nil
AbtOracleDatabaseConnection(AbtDatabaseConnection)>>#disconnect
   self=an AbtOracleDatabaseConnection
...
==============<STACK TRACE ENDS>=============


5. Resource leak if connect fails

It seems that you were too impatient. :wink: I ran a test and found that the problem occurs after 1380 iterations.

Also, you can watch
Code: Select all
AbtThreadManager default processDictionary size

growing.

6. NUMBER(x,0) is handled as a scaled decimal, not as an integer

I have not declared my fields as smallint or int, I am using fields of type NUMBER(x,0) only.

So which Smalltalk number classes did you see being used for smallint, int and number(x,0) in your tests? Did you test with the AbtOracle* or the AbtOracle8* classes?

10. Stored procedure parameter lists hold references to freed OS objects after disconnect

Are you sure that you reused the same AbtStoredProc instance? The problem does not occur if you use a new instance. Did you test with the AbtOracle* or the AbtOracle8* classes?

Thanks and best regards,
Christoph
cknittel
 
Posts: 27
Joined: Tue Nov 28, 2006 6:10 am
Location: Vienna - Austria

AbtDatabaseManager class>>#libraryName

Postby cknittel » Tue Dec 12, 2006 7:04 am

Hi tc,

Another issue:

12. AbtDatabaseManager class>>#libraryName should be changeable by the user without subclassing

Currently, AbtDatabaseManager class>>#libraryName is implemented like this:

Code: Select all
libraryName
   "Answer the library name for the current environment."

   ^self selectLibraryName: self libraryNameTable


And the class method libraryNameTable is implemented like this in AbtOracleDatabaseManager:

Code: Select all
libraryNameTable
   "..."

   ^#(
      ('OS/2' 'ora_d73o')
      ('WIN32*' 'ora73win.dll' )
      ('AIX' 'libclntsh' )
      ('HP-UX' 'libclntsh.sl' )
      ('SunOS' 'libclntsh.so' )
   )


As mentioned previously, the AbtOracle* classes, though intended for Oracle 7, work perfectly with newer Oracle versions. For that, however, the library name (for the Windows platform) must be changed to "oci.dll" (as it is in AbtOracle8DatabaseManager).

Therefore a simple way for changing the library name should be provided.

For example, the existing method #libraryNameTable could be renamed to #defaultLibraryNameTable, and a new #libraryNameTable method could be added as an accessor for a class instance variable "libraryNameTable" (which would be initialized using #defaultLibraryNameTable).

Thanks and best regards,
Christoph
cknittel
 
Posts: 27
Joined: Tue Nov 28, 2006 6:10 am
Location: Vienna - Austria

AbtOracle8* LONG (RAW) Issue

Postby cknittel » Tue Dec 12, 2006 7:49 am

Hi again,

I just checked what the issue with the AbtOracle8* classes was and found that:

- Fetching data from a table containing a LONG/LONG RAW column gives me an ORA-01062 followed by a General Protection Fault.

- The problem is caused by AbtOracleLongField>>#oracleBindLength.

Code: Select all
oracleBindLength
   " Return the length of this field when used as a host variable parameter
     in a SQL statement.  LONG is special case.  -1  indicates that length is stored
     in first 4 bytes of the field "

   ^self length > 65300
      ifTrue: [ -1 ]
      ifFalse: [ self length ]


If you check the senders, you will find that there are several #allocateAndPrepareOSMemoryFor... methods, and that these would actually need the implementation to be

Code: Select all
oracleBindLength
   "..."

   ^self length


However, this wouldn't work with the other senders (#initializeColumnLengths, #initialize:) which expect a short field.

I tried to use two different "length" methods for the two situations, but it never worked completely in all cases, and so I finally gave up. :(

Best regards,
Christoph
cknittel
 
Posts: 27
Joined: Tue Nov 28, 2006 6:10 am
Location: Vienna - Austria

Postby tc » Tue Dec 12, 2006 11:17 am

Were you using 'long raw' as a BLOB column?

--tc
tc
Moderator
 
Posts: 304
Joined: Tue Oct 17, 2006 7:40 am
Location: Raleigh, NC

LONG RAW

Postby cknittel » Wed Dec 13, 2006 6:39 am

Hi tc,

Yes, I was using 'long raw' as a BLOB column. But I think the problem is not related to whether the column contains ascii or binary data.

I am looking forward to LOB support in VA 7.5! :-)

Please let me know if you need further information regarding the other items.

Thanks and best regards,
Christoph
cknittel
 
Posts: 27
Joined: Tue Nov 28, 2006 6:10 am
Location: Vienna - Austria

11.) Raising exceptions in an error block causes cursor leak

Postby cknittel » Tue Feb 06, 2007 5:51 am

Hi,

It seems that some of the fixes made it into VA 7.5! Thanks!

Some more thoughts regarding item

11.) Raising exceptions in an error block causes cursor leak

In the current design, "self errorBlock" is actually invoked in the outermost methods only (AbtStoredProc>>#invoke, AbtDatabasePart>>#commitTransaction, AbtDatabaseConnection>>#disconnect, ...).

The error block is then passed on to other methods as a parameter (#...ifError:).

E.g. in AbtStoredProc>>#invoke:

Code: Select all
...
AbtDbmSystem execLongDatabaseOperation: [
   ( connection  := self activateDatabaseMgr ) abrIsDatabaseConnection
      ifTrue: [ 
         rc := ( connection 
            invokeStoredProcedureNamed: self procedureSpec procName
               inputParameters: self inputs
               outputParameters: self outputs
               inputString: self inputString
               useSqlCall: self procedureSpec useSqlCall
               ifError: self errorBlock ).
         ...
         ] 
   ].


If this was changed to

Code: Select all
| error |
error := nil.
...
AbtDbmSystem execLongDatabaseOperation: [
   ( connection  := self activateDatabaseMgr ) abrIsDatabaseConnection
      ifTrue: [ 
         rc := ( connection 
            invokeStoredProcedureNamed: self procedureSpec procName
               inputParameters: self inputs
               outputParameters: self outputs
               inputString: self inputString
               useSqlCall: self procedureSpec useSqlCall
               ifError: [ :err | error := err. ] ).
         ...
         ] 
   ].

error notNil ifTrue: [ self errorBlock value: error. ].


it would be exception-safe.

This might be an easier solution than ripping the "ifError:" out from every method and freeing resources using #ensure:. However, it would still mean that a lot of methods would have to be changed.


While playing around with this idea, I noticed that there are a few #...ifError: methods that do use "self errorBlock" instead of the error block that was parsed as an argument:

Code: Select all
AbtOdbcDatabaseConnection>>#foreignKeyReferencesForTableNamed:ifError:
AbtOdbcDatabaseConnection>>#foreignKeysOfTableNamed:ifError:
AbtOdbcDatabaseConnection>>#getInfoIfError:
AbtOdbcDatabaseConnection>>#primaryKeyOfTableNamed:ifError:
AbtOracle8DatabaseConnection>>#prepareSQLStatement:sqlStmt:ifError:


(and maybe others?). IMO this is a bug and should be fixed in the next maintenance release.

Thanks and best regards,
Christoph
cknittel
 
Posts: 27
Joined: Tue Nov 28, 2006 6:10 am
Location: Vienna - Austria

Postby tc » Mon Feb 12, 2007 11:15 pm

Hello,

Yes, I agree, not sure which release it will be in.

Thanks for your posts.

--tc
tc
Moderator
 
Posts: 304
Joined: Tue Oct 17, 2006 7:40 am
Location: Raleigh, NC

Next

Return to VA Smalltalk 7.0, 7.5 & 8.0

Who is online

Users browsing this forum: No registered users and 1 guest