I always like to try out and look at new releases of databases. Since Oracle is my forte this happens to be the new version, Oracle 11g.
Now that it is available for free download for both Windows and Linux, it is a good time to learn all about the new features as well as to experiment and see what works well and what is still a new undocumented feature or bug. Here
is a useful link on new features for Oracle 11G.
I plan to blog more on the most useful new features in the future for 11g and Oracle.
Monday, October 29, 2007
Saturday, April 14, 2007
Why a Developer DBA adds value to the organization
Recently have considered the whole discussion surrounding the DBA and Developer relationship.
Typically most IT shops have a developer team that manages the development and test environments and a senior level DBA to focus primary on production mission critical database and applications. However, as the reality sets in a firm's development arean can be extremely critical if it ever goes down and it does because development and test systems tend to be more flakey and less stable than production as changes are made on a frequent basis, tables and schema changes occur on the fly and if the development application and database is not available then the developers cannot get any work done which greatly upsets the status quo of things.
A good discussion of the purpose for a development DBA can be found here
http://www.simple-talk.com/sql/database-administration/what-use-is-a-development-dba/
Typically most IT shops have a developer team that manages the development and test environments and a senior level DBA to focus primary on production mission critical database and applications. However, as the reality sets in a firm's development arean can be extremely critical if it ever goes down and it does because development and test systems tend to be more flakey and less stable than production as changes are made on a frequent basis, tables and schema changes occur on the fly and if the development application and database is not available then the developers cannot get any work done which greatly upsets the status quo of things.
A good discussion of the purpose for a development DBA can be found here
http://www.simple-talk.com/sql/database-administration/what-use-is-a-development-dba/
Saturday, April 7, 2007
Thai beer, writing and learning
Its been a crazy 4 weeks with new work on database technologies, writing a book on database migrations to Oracle 10g, and meeting new colleagues and associates.
While not a big consumer of alcoholic beverages, I can appreciate the benefit of washing down spicy foods with Thai beer. Of course, one of my new colleagues has the amazing talent of not only smoking like a chimney (8 packs of cigarettes a day!) but can eat 2 sushi combination dinners and drink between 3-8 beers at one sitting!! I am amazed at how some folks have god-like genetics and can do such feats of hard living.
I have been consumed with so many things lately between work, travel, and professional development that its a good busy schedule.
While not a big consumer of alcoholic beverages, I can appreciate the benefit of washing down spicy foods with Thai beer. Of course, one of my new colleagues has the amazing talent of not only smoking like a chimney (8 packs of cigarettes a day!) but can eat 2 sushi combination dinners and drink between 3-8 beers at one sitting!! I am amazed at how some folks have god-like genetics and can do such feats of hard living.
I have been consumed with so many things lately between work, travel, and professional development that its a good busy schedule.
Sunday, February 25, 2007
Crazy neighbors and a bottle of beer
Last night at aproximately 12:25am while I was sound asleep, a knock came on my front door.
Since I was half asleep and do not answer my door to unknown strangers, I went back to bed. The man was persistent but finally my ability to pass into deep slumber won the contest.
This morning after waking and taking out the trash, I found the following item on my doorstep:
Since I was half asleep and do not answer my door to unknown strangers, I went back to bed. The man was persistent but finally my ability to pass into deep slumber won the contest.
This morning after waking and taking out the trash, I found the following item on my doorstep:
Yuck! I am not a big fan of beer begin with and the very notion of fruit beer makes me want to vomit.
Since I rarely if ever consume alcoholic beverages (booze destroys brain cells, pickles the liver, and helps make one obese), I disposed of the bottle in the local trash receptacle.
Wednesday, February 14, 2007
ETL design and methods revisited
The theme is ETL tools and ETL design patterns:
I. ETL methodology
Current data warehouse and ETL architects and developers have devised numerous theories and practices covering ETL design and support. Kimball has an excellent analysis of the ETL process from an architect role in his excellent text, The Data Warehouse ETL Toolkit. In a nutshell, performance issues can be addressed in a couple basic ways:
1. Drop or disable primary key constraints and indexes before load and transformations are performed.
2. Separate UPDATE and INSERT transformations to optimize ETL processing.
To build on the last discussion concerning ETL design methods and tools, I would like to analyze various offerings from home grown scripts to vendor tools. The major players in the current ETL market:
Informatica
GoldenGate
Business Objects
Microstrategy
In my last discourse on ETL design, I took the devils advocate and offered pros and cons on home grown ETL ie) roll your own versus implementation with a robust ETL tool such as Informatica PowerCenter. With the reality of heterogeneous mixed environments and many different source systems, it is not realistic to use home grown ETL unless one has a very large and highly skilled staff of developers and DBAs UNLESS one has a single platform such as Oracle on UNIX or Microsoft SQL Server on Windows (which can use the built in ETL tool called DTS). Most shops would actually be better off using a modern time tested and robust tool such as Informatica or another tool such as Business Objects or Golden Gate ETL software.
I. ETL methodology
Current data warehouse and ETL architects and developers have devised numerous theories and practices covering ETL design and support. Kimball has an excellent analysis of the ETL process from an architect role in his excellent text, The Data Warehouse ETL Toolkit. In a nutshell, performance issues can be addressed in a couple basic ways:
1. Drop or disable primary key constraints and indexes before load and transformations are performed.
2. Separate UPDATE and INSERT transformations to optimize ETL processing.
To build on the last discussion concerning ETL design methods and tools, I would like to analyze various offerings from home grown scripts to vendor tools. The major players in the current ETL market:
Informatica
GoldenGate
Business Objects
Microstrategy
In my last discourse on ETL design, I took the devils advocate and offered pros and cons on home grown ETL ie) roll your own versus implementation with a robust ETL tool such as Informatica PowerCenter. With the reality of heterogeneous mixed environments and many different source systems, it is not realistic to use home grown ETL unless one has a very large and highly skilled staff of developers and DBAs UNLESS one has a single platform such as Oracle on UNIX or Microsoft SQL Server on Windows (which can use the built in ETL tool called DTS). Most shops would actually be better off using a modern time tested and robust tool such as Informatica or another tool such as Business Objects or Golden Gate ETL software.
Wednesday, January 31, 2007
Lack of Critical Thinking and Reasoning Skills
Well folks before I switch to a more technical series of posts, I thought that I would submit a rant.
I have noticed lately in the last few years a dire lack of critical reasoning and thinking skills in the technical arena. Maybe it is due to the "get it down quick not good" attitude of management or maybe its lack of real talent. Who knows? But to me it is a pet peeve when a person fails to challenge himself/herself and gives up all too easy when presented with a problem. To me, one should take a deep breath, relax or chill and take a fresh simplified approach to solving a problem. Perhaps its fear of failure or inner doubt? When a senior or even a junior database or other technical person could just do a simple google search or lookup the question in a manual, it would save time and frustration. Case in point, I have seen 'supposed' senior level database developers unable to write a simple JOIN query let alone a nested subquery in Oracle!!! While I do not profess to be a developer, even I can write basic SQL.
It is far better to take the time to do it right the first time rather than to waste time spinning wheels over a poor and rushed solution.
Case in point, recently on my current project there has been performance problems that have been occurring on daily basis in afternoons for the database servers. Initially the manager/DBA lead assumed that it was virus scanner software that was causing the performance issue. I kindly replied that we cannot assume this and need to conduct tests to discover the true culprit. Sure enough, my hunch was correct and it turns out to be a shortage of memory and CPUs on the server. The virus scanner software took little resources maybe 0.0002 per cent of usage on the server. I did find one user, a Cognos developer, nonetheless who was running a resource intensive report that uses full outer joins to be consuming 25% of the server resources.
I have noticed lately in the last few years a dire lack of critical reasoning and thinking skills in the technical arena. Maybe it is due to the "get it down quick not good" attitude of management or maybe its lack of real talent. Who knows? But to me it is a pet peeve when a person fails to challenge himself/herself and gives up all too easy when presented with a problem. To me, one should take a deep breath, relax or chill and take a fresh simplified approach to solving a problem. Perhaps its fear of failure or inner doubt? When a senior or even a junior database or other technical person could just do a simple google search or lookup the question in a manual, it would save time and frustration. Case in point, I have seen 'supposed' senior level database developers unable to write a simple JOIN query let alone a nested subquery in Oracle!!! While I do not profess to be a developer, even I can write basic SQL.
It is far better to take the time to do it right the first time rather than to waste time spinning wheels over a poor and rushed solution.
Case in point, recently on my current project there has been performance problems that have been occurring on daily basis in afternoons for the database servers. Initially the manager/DBA lead assumed that it was virus scanner software that was causing the performance issue. I kindly replied that we cannot assume this and need to conduct tests to discover the true culprit. Sure enough, my hunch was correct and it turns out to be a shortage of memory and CPUs on the server. The virus scanner software took little resources maybe 0.0002 per cent of usage on the server. I did find one user, a Cognos developer, nonetheless who was running a resource intensive report that uses full outer joins to be consuming 25% of the server resources.
Wednesday, January 24, 2007
Real World DBA experience versus OCP certification: Which is better?
I have been mulling over the whole value of vendor based certifications as of late such as the OCP, mcDBA and others. With the exception of the OCM "Oracle Certified Masters", my personal view is that real experience trumps any certification hands down. Alas many companies have a biased view that an OCP is more valuable than hands on experience. The only case that I would respect an OCP nowadays is IF the person has both the OCP and years of real DBA experience. Like for example, if I had to interview two candidates and one had the OCP and same experience as the other candidate then the OCP would help to be a tie-breaker. However if the candidate had superior technical experience/skills and soft skills that counts for more than an OCP in my book. One issue is that the 10g OCP only requires 2 exams to pass so it is really a more "watered" down version of the older 9i OCP program that required 4 exams to be passed for the certification. Fortunately, I am in good company with many top industry experts on the relative merit or lack thereof in terms of how valued the OCP is:
Dizwell view of OCP
Tom Kyte's view of OCP
Dizwell view of OCP
Tom Kyte's view of OCP
Tuesday, January 16, 2007
What is a good manager and a good client and company?
Some excellent discussion on
management in the IT profession.
management in the IT profession.
Happy DBA birthday to a DBA!
Well its a new day new year and my 36th birthday! So guess I will have to do something special to celebrate.
It is a bummer that I have to work today but heck should be fun- we have disaster recovery testing this week for Oracle and SQL Server environments and it will be fun to refresh my DR and Backup/Recovery knowledge and find clever ways to enlighten the team on the process. I would like to find a way to convince the management team to implement a hot standby site for Oracle and SQL Server with Data Guard and to design the process. Would help me sleep better at night as a DBA.
On a side note: communication is the sticky point in human relationships- often in written, verbal and nonverbal communications the misunderstandings create conflict among human beings. I strive to be the ultimate communicator and build the trust and human relationships that foster good energy and harmony.
Case in point- recently I had a discussion with a fellow Oracle professional and did my best to reassure him that I meant no ill will. As he is a consumate Oracle guru I let him know that my intent is sincere and pure. It really is like that old Genesis song when Phil Collins sang about "Misunderstanding"
It is a bummer that I have to work today but heck should be fun- we have disaster recovery testing this week for Oracle and SQL Server environments and it will be fun to refresh my DR and Backup/Recovery knowledge and find clever ways to enlighten the team on the process. I would like to find a way to convince the management team to implement a hot standby site for Oracle and SQL Server with Data Guard and to design the process. Would help me sleep better at night as a DBA.
On a side note: communication is the sticky point in human relationships- often in written, verbal and nonverbal communications the misunderstandings create conflict among human beings. I strive to be the ultimate communicator and build the trust and human relationships that foster good energy and harmony.
Case in point- recently I had a discussion with a fellow Oracle professional and did my best to reassure him that I meant no ill will. As he is a consumate Oracle guru I let him know that my intent is sincere and pure. It really is like that old Genesis song when Phil Collins sang about "Misunderstanding"
Saturday, January 13, 2007
What is a DBA?
From Wikipedia online Encyclopedia website as to what constitutes an
Database Administrator:
A database administrator (DBA) is a person who is responsible for the environmental aspects of a database. In general, these include:
Recoverability - Creating and testing Backups
Integrity - Verifying or helping to verify data integrity
Security - Defining and/or implementing access controls to the data
Availability - Ensuring maximum uptime
Performance - Ensuring maximum performance given budgetary constraints
Development and testing support - Helping programmers and engineers to efficiently utilize the database.
The role of a database administrator has changed according to the technology of database management systems (DBMSs) as well as the needs of the owners of the databases. For example, although logical and physical database design are traditionally the duties of a database analyst or database designer, a DBA may be tasked to perform those duties.
However, there is a real crisis with the role of DBA in the USA and world. Often a DBA is considered to be a catch all phase for anyone who works with a database. That makes the whole situation sticky and ambiguous as to how one becomes and remains employed as a DBA! Below are some interesting links to the whole confusion around DBA and Developers:
Oracle WTF view of Oracle DBA:
More about what is a DBA
The Oracle Alchemist Discussion between DBA versus Developer
I personally have been more of the traditional production operational DBA although have done some development work for clients in the past. Currently I am working on revamping the large SQL Server database and Oracle environment to replace the legacy scripts with automated Informatica, SQL, PL/SQL and T-SQL scripts and workflows. Since the developers are not only very junior level but more C++/Java guys with limited SQL experience and zero Informatica skills, the team has involved me heavily in design of the data warehouse models. Its fun and challenging. Personally I'd rather just script everything in SQL, Transact SQL and PL/SQL and say to hell with it with the overpriced and bloated POS tool called Informatica Power Center but management has already invested eh sunk a million bucks into the purchase of this tool. Funny- it costs a fortune to create a data warehouse and ETL process with these fancy shmancy tools whereas if the time and care had been spent writing clean SQL and PL/SQL code it would have saved this healthcare place a fortune!
Database Administrator:
A database administrator (DBA) is a person who is responsible for the environmental aspects of a database. In general, these include:
Recoverability - Creating and testing Backups
Integrity - Verifying or helping to verify data integrity
Security - Defining and/or implementing access controls to the data
Availability - Ensuring maximum uptime
Performance - Ensuring maximum performance given budgetary constraints
Development and testing support - Helping programmers and engineers to efficiently utilize the database.
The role of a database administrator has changed according to the technology of database management systems (DBMSs) as well as the needs of the owners of the databases. For example, although logical and physical database design are traditionally the duties of a database analyst or database designer, a DBA may be tasked to perform those duties.
However, there is a real crisis with the role of DBA in the USA and world. Often a DBA is considered to be a catch all phase for anyone who works with a database. That makes the whole situation sticky and ambiguous as to how one becomes and remains employed as a DBA! Below are some interesting links to the whole confusion around DBA and Developers:
Oracle WTF view of Oracle DBA:
More about what is a DBA
The Oracle Alchemist Discussion between DBA versus Developer
I personally have been more of the traditional production operational DBA although have done some development work for clients in the past. Currently I am working on revamping the large SQL Server database and Oracle environment to replace the legacy scripts with automated Informatica, SQL, PL/SQL and T-SQL scripts and workflows. Since the developers are not only very junior level but more C++/Java guys with limited SQL experience and zero Informatica skills, the team has involved me heavily in design of the data warehouse models. Its fun and challenging. Personally I'd rather just script everything in SQL, Transact SQL and PL/SQL and say to hell with it with the overpriced and bloated POS tool called Informatica Power Center but management has already invested eh sunk a million bucks into the purchase of this tool. Funny- it costs a fortune to create a data warehouse and ETL process with these fancy shmancy tools whereas if the time and care had been spent writing clean SQL and PL/SQL code it would have saved this healthcare place a fortune!
Thursday, January 11, 2007
ETL Tools versus "home grown ETL"
Well I have mixed views of fancy vendor based ETL and Business Intelligence Tools "BI" vendors such as Informatica and Cognos. In theory, these commercial off the shelf "COTS" based data warehouse tools sound like a panacea for busy development and production data warehouse teams who have the daunting task of building and deploying a potent data warehouse in crunch times from mixed data sources and technologies. Lets suppose that we have 200 servers with Oracle, Sybase, and Microsoft SQL Server databases running on Windows, UNIX and main frame based servers and 20 different data source feeds. To build the ideal home grown ETL environment is labor intensive and requires senior level technical staff. In the modern buzz words that software vendors mesmerize executive management professionals, the Informatica and Cognos tools sound like a perfect fix. "Oh we can build and deploy a full data warehouse with complex ETL logic in a few months!". Several million dollars later and months of delays and broken ETL pipelines, the costs of expensive ETL consultants has frustrated many DBA and developer staff not to mention inciting the wrath of IT managers.
Here is a good link to a relevant discussion on commercial ETL tools and home grown solutions:
ETL Tools Roundtable
Recap of home grown ETL versus commercial ETL tools and vendors
In fact many industry experts argue the case against ETL tool vendors and favor a custom home grown design with SQL and PL/SQL scripts for Oracle instead of the head aches of expensive tools such as Informatica. My beef with Informatica and other ETL vendors is that the tool hides the workings of the database and ETL processes. It requires expensive learning curve and delays. And even worse- the whole tool is a third party propiety method that is not based on a portable industry standard such as ANSI SQL. Heck, its bad enough that I must port Transact SQL code to Oracle PL/SQL. Lest not burden the poor overworked DBA with yet another messy tool!
Here is a good link to a relevant discussion on commercial ETL tools and home grown solutions:
ETL Tools Roundtable
Recap of home grown ETL versus commercial ETL tools and vendors
In fact many industry experts argue the case against ETL tool vendors and favor a custom home grown design with SQL and PL/SQL scripts for Oracle instead of the head aches of expensive tools such as Informatica. My beef with Informatica and other ETL vendors is that the tool hides the workings of the database and ETL processes. It requires expensive learning curve and delays. And even worse- the whole tool is a third party propiety method that is not based on a portable industry standard such as ANSI SQL. Heck, its bad enough that I must port Transact SQL code to Oracle PL/SQL. Lest not burden the poor overworked DBA with yet another messy tool!
Sunday, January 7, 2007
New Year 2007
Well folks,
A new year is now upon us with many changes to come of the good kind, I hope.
I am working on a project for a large healthcare firm to perform database migrations from SQL Server 2000 to Oracle 10g Release 2. The client also has me working with their team of ETL developers to create new mappings and workflows in an ETL tool called Informatica Power Center version 8. The current production, development, test environment is running on Microsoft SQL Server 2000 database on Windows 2000 Server. Management has goal to migrate completely off of the Microsoft platform to Oracle 10g over a 2 year time plan.
A new year is now upon us with many changes to come of the good kind, I hope.
I am working on a project for a large healthcare firm to perform database migrations from SQL Server 2000 to Oracle 10g Release 2. The client also has me working with their team of ETL developers to create new mappings and workflows in an ETL tool called Informatica Power Center version 8. The current production, development, test environment is running on Microsoft SQL Server 2000 database on Windows 2000 Server. Management has goal to migrate completely off of the Microsoft platform to Oracle 10g over a 2 year time plan.
Subscribe to:
Posts (Atom)
