UPDATE T1 
INNER JOIN (T2 INNER JOIN T3 ON (T2.C1 = T3.C1) AND (T2.C2 = T3.C2)) ON (T1.C1 = T2.C1) AND (T1.C2 = T2.C2) 
SET T3.V1 = T1.V1
WHERE T3.C3 = 'X' AND T2.C3 = 'Y' AND T1.C3 = 'Z';

Looking at the SharedServices_Security.log, the following error is listed:

2010-03-22 13:58:56,215 [main] WARN com.hyperion.css.CSSAPIImpl.initialize(Unknown Source) - Unable to communicate with the server.

Error Code: -1
com.hyperion.css.CSSCommunicationException: LDAP porvider could not be contacted. Error Code: 3

Googling here and there, I came across this valuable article by the Hyperion guru John Goodwin.  The article talks about recovering the Shared Services OpenLDAP database using db_recover, which I expected to solve my problem.

I just faced one problem, and that was when I tried to execute the db_recover command, as mentioned by John in his blog. The command gave an error that the libdb-4.3.so file is missing. I just had to append the lib directory, which is [HYPERION_HOME]/SharedServices/9.3.1/openLDAP/usr/local/bdb/lib, to the LIBPATH.

Problem resolved Thanks John.

1. Try to fix the corrupt block (okay, if the blocks are not zero-level blocks)
2. Try to restore the DB from a previous backup (ideal, if the backup is a very recent one)
3. Create a new application database from scratch, losing all data you had on the old one (that’s the disastrous one)

Of course, the third option was the first option to drop.  The second option, is the ideal one, but not in our case since the last backup we took was not a complete one!  So, we had to go with the first – that is why I put it as first option

Okay, the term fix here does not really mean that it will correct the invalid block headers.  It actually means that any block with an IBH will be removed.  Nice, huh?  In fact, that’s okay if the block is a high-level block (parent block); if it gets removed, it can always be recalculated based on the underlying children blocks.  A bad situation would be if corrupt blocks were zero-level blocks; if they get removed, they cannot be recalculated, and you will have to create them by loading data again to create those zero-level blocks.  The bad news was there were corrupt zero-level blocks.  The good news, they were not critical data blocks, so it was okay to remove them.

Now, how do you do the fix?  According to Oracle metalink (support), there are different methods.  One is to use MaxL commands, which is the method we used.  Funnily, the article posted on Oracle support had syntax errors in the MaxL commands, but Essbase documentation had it right.  Anyways, you do this in 4 steps:

1. Disable all logins.
2. Forcibly log off all users.
3. Run the MaxL statement to get invalid block header information.
4. Repair invalid block headers, if applicable.

It is important that no Essbase transactions are taking place while you are doing this.  That is why it is important to do the first 2 steps.  The third step produces a text file containing all member combinations that yield corrupted blocks.  This how you will know if the blocks were zero-level or parent-level.  The third step repairs the block headers, by removing the blocks with IBH.

So, for example, assuming sample is our application name, and basic is our database name, run the following MaxL commands for steps 1 and 2:

alter application sample disable connects;
alter system logout session on database sample.basic;

When done successfully, run the following command for step 3:

alter database sample.basic validate data to local logfile 'invalid_blocks';

The file name invalid_blocks between single-quotes is the output text file, which will contain the corrupted blocks.  It will be located under your user home folder, if you are using a *nix (UNIX, Linux) system.

Note: When executing the third command above, which is the validate command, and you get an error saying “Aborted due to status 1130610“, then probably you have attempted to run the third command and aborted it unexpectedly.  To resolve this error, you have to delete a file called ibh.lst under [HYPERION_HOME]/AnalyticServices/app/[application name]/[database name] and try again, if you are using a *nix system.

Once validation is done, and you have checked the output file and are aware of the blocks that will be removed, you can go ahead with the following command for step 4:

alter database sample.basic repair invalid_block_headers;

Now, the last step is to recalculate the parent blocks that were removed.  If your situation was a worst case scenario and there were zero-level blocks that had to be removed for repair, then you will need to restore those first (e.g. using data load) and then recalculate dependent parent blocks.

        int[] result = new int[array.length];

        for(int i = 0, top = 0, bottom = array.length - 1; i < array.length; i++) {
            if(array[i] == 0) {
                result[top] = array[i];
                top++;
            } else {
                result[bottom] = array[i];
                bottom--;
            }
        }

// Get the class you want to reflect
            Class reflectedClass = Class.forName("tsoliman.wordpress.com.MyReflectionClass", true, ClassLoader.getSystemClassLoader());
            
            // Create an instance of reflectedClass
            Object reflectedClassObj = reflectedClass.newInstance();
            
            // Get the method you want to invoke
            Method reflectedMethod1 = reflectedClass.getMethod("myReflectionMethod");
            
            // Prepare argument types and values
            Class[] argsType = {String.class, String.class};
            String[] methodArgs = {"Foe unvanquished, I won't perish in the field; ", "I'll be born again, to take up the halberd seven more times."};

            // Invoke the method with no arguments on reflectedClassObj
            reflectedMethod1.invoke(reflectedClassObj, (Object[])null);

            // Invoke the method with 1 String argument on reflectedClassObj
            Method reflectedMethod2 = reflectedClass.getMethod("myReflectionMethod", argsType[0]);
            reflectedMethod2.invoke(reflectedClassObj, (Object)methodArgs[0]); // Casting using Object to avoid compiler warning

            // Invoke the method with 2 String arguments on reflectedClassObj
            Method reflectedMethod3 = reflectedClass.getMethod("myReflectionMethod", argsType);
            reflectedMethod3.invoke(reflectedClassObj, (Object[])methodArgs); // Casting using Object[] to avoid compiler warning

    public static void doDFS(MyVertex v, ArrayList visited) {
        visited.add(v);
        
        preorderProcess(v);
        
        ArrayList neighborsArray = v.getNeighbors();
        if (null != neighborsArray) {
            Enumeration neighborsEnum = Collections.enumeration(neighborsArray);

            while (neighborsEnum.hasMoreElements()) {
                doDFS((MyVertex) neighborsEnum.nextElement(), visited);
            }
        }
        
        postorderProcess(v);
    }

Here are a few tips that might help you resolve your issue:

1. Get an LDAP browser; the one I use is a free and relatively easy to use LDAP browser called JXplorer
2. Start your LDAP browser, fill in the connection details with the following values:
   Host: Your Shared Services host name or IP
   Port: 58089
   Protocol: LDAP v3
   User DN: cn=911,ou=People,dc=css,dc=hyperion,dc=com (911 is for admin user)
   Password: (admin password; default is password)
   A connection dialog should look like this:
   

Now you should be able to see the full LDAP hierarchy; i.e. Groups, Users, Projects, Roles, etc.

It’s just a matter of time.