Monday, 29 December 2014
While focusing on management skills has given me some great insights and helped me develop some invaluable skills, I feel its time to turn my attention back to some technical challenges.
Luckily as I (as of a few months ago) now work for one of the world's best BI vendors (more on this soon), I have plenty of that on the horizon !
Friday, 17 October 2014
You can do a full MBA if you are a glutton for punishment. For me, as I work fulltime (and then some) as well as having a wife and two young kids, I found it too great a burden time wise to go beyond the graduate certificate (at least for now) so I only did the graduate certificate.
Details of what I did are here ( scroll down to the graduate certificate section) : Graduate Certificate in Management (Information Technology) . It was well worth doing and although it took me way out of my comfort zone, it was extremely valuable and I highly recommend it !
This is not the only change i'm making however - I've decide to go back to consulting. It will be great to be back at the coalface (Ineed a more environmentaly friendly metaphor !) helping clients across Asia Pacific.
More details to come !
Tuesday, 14 January 2014
I think you will need access to the sap portal to view it, but the release notes are here:
DS 4.1 SP3
Now to shake of the Holiday cobwebs and get cracking with 2014 !
Tuesday, 24 December 2013
Tuesday, 17 December 2013
I came across a good website yesterday for understanding this. I thought i'd share it here for others:
I hope this helps you !
Wednesday, 4 December 2013
Source System knowledge: Due to the fact that we use big ERP/SAAS solutions like SAP and SFDC, out business don't have the technical knowledge of where their data comes from in the underlying systems. This means we have needed to build this knowledge up in our team, which takes a lot of time (We don't have the budget to bring in experts).
Getting to the data - 2 main areas this hits us - Cloud solutions that don't provide adequate interfaces for getting to their data for BI & SAP ERP, which prohibits getting data more than once a day for some key areas (unless you buy Hana for $$$$). Its almost as if these vendors want you to be locked-in to their solution !
Bad data - a classic, but still an issue.
Design - I'm starting to think Star schema's and particularly slowly changing dimensions are agile handbrakes.. The amount of work that goes into adding fields to SCD2 tables and not having it generate a new generation requires a lot of extra coding.
These are only a few, but hopefully someone has a solution - and I don't mean a new tool - I am really trying to find a better process, if there is one !
Look forward to hearing your thoughts
Tuesday, 3 December 2013
Thanks David !
If We Build It, Will They Come?
Tuesday, 8 October 2013
To me, I don't do 40+ hours a week 48 weeks a year very well. It burns me out. For some people that may be perfect, but not me. One day I hope to find an employer that lets me work 4 days a week or 46 weeks a year (or both). I think that would be about perfect.
Has anyone found the perfect balance ? Are there any employers out their who let their employees pick the conditions that suit them ?
Let me know if you've found the perfect balance !
Thursday, 15 August 2013
We recently moved out repositories from one SQL server to another. We have no end of problems and are still finding little issues. So I thought it would help others if I posted a few tips:
- Before starting, make sure everything is checked into the central repository.
- For local repositories, you are better off to create new one's on the destination server, and remove the old ones.
- Ensure that you have everything correct in IPS/BO before worrying about the job server designer.
- If you get errors trying to add repositories into the job server, you will need to change the job server name. One the repos are added, just change it straight back. This seems to be a bug in the job server manager.
I hope these help ! If I knew this before we stared, I would have had a much easier few weeks !
Tuesday, 12 February 2013
Friday, 11 January 2013
I have been trying to compare the pros and cons of moving to tabular for my company. I wanted to be able to look at them side my side, but it wasn't obvious how to. Luckly someone else worked it out for me:
Big thanks to James Serra for working this out !
The patches are compressed with WinRAR. When you double click on the patch, it decompresses it to your temporary directory. In our case that was on the C: drive, but the server in question did not have enough room on the C: drive. This resulted in a corrupted install of data services - not good !
To fix this, I did the following:
Open a command prompt as administrator.
Manually set the TMP and TEMP variables to point to another location (in out case f::\temp). I then ran the patch executable from this command prompt session. This successfully uncompressed. I then did a 'repair' of my data services installation which fixed the problem.
I hope this saves you from having the same issue !
What will 2013 bring ? Will the big data hype settle down. Will the NO-SQL crowd be subsumed by the NEW-SQL crowd ? Will the industry come up with some more buz-words (That one's a definite !)
Anyway, happy new year to my reader(s) !
Friday, 5 October 2012
Summary - you don't need big data unless you know you need big data.
Thursday, 6 September 2012
- Data Services WIKI
- Data Integrator @ Business Objects Board
- Data Services @ SAP Community Network (SCN)
- ETL Doctor
Friday, 30 March 2012
Installing SAP Data Services 4.0 : Part 1 - Information Platform Services
Installing SAP Data Services : Part 2 - Data Services
Wednesday, 21 March 2012
Anyway, I've been deep into SAP BO Data Services 4 and am really enjoying working with the product. I hope to make some more focused posted around this soon.
Hopefully not in another 6 months...
Friday, 11 November 2011
I was fortunate enough recently to attend the DW 2.0 Conference in Sydney. One of the presenters, Therese Ahlstam (from the Genesee Academy) advocated a technique for building data warehouse's she referred to as Hyper-agility. The idea was to avoid the need to constant schema changes by storing Staging & ODS tables as Key/Value pairs. To me this seems to fit the Serialisation/de-serialisation design pattern so common in many software frameworks these days.
This idea seemed quite radical to me, but I've been thinking on this ever since and I decided to have a look at how feisible this really is. As it turns out, while the syntax is a bit tricky, this is not only possible but relatively easy to do in MS SQL Server thanks to the PIVOT/UNPIVOT commands.
Here is an example:
CREATE TABLE SourceTable
INSERT INTO SourceTable VALUES (1,'Bob Smith', '1 Miller St Sydney', '555-1234');
INSERT INTO SourceTable VALUES (2, 'John wayne', '10 california Dr Hollywood', '+1-555-4567');
INSERT INTO SourceTable VALUES (3,' Billy Bowden', '4 Wainui road Wellington', '+64-21-3456789');
--Unpivot the table.
SELECT pkID, KeyCol, ValCol INTO unpivotSourceTable
(SELECT pkID, CAST(Name as VARCHAR(250)) as Name,Address,cast(Phone as VARCHAR(250)) as Phone from SourceTable) ST
(ValCol FOR KeyCol IN (Name,Address,Phone)
select * from unpivotSourceTable
ORDER BY pkID, KeyCol
pkID KeyCol ValCol
1 Address 1 Miller St Sydney
1 Name Bob Smith
1 Phone 555-1234
2 Address 10 california Dr Hollywood
2 Name John wayne
2 Phone +1-555-4567
3 Address 4 Wainui road Wellington
3 Name Billy Bowden
3 Phone +64-21-3456789
Note we need to convert all the columns to the same data type to get this to work (which makes sense) . This is how we would store the data in our Operational data store and/or staging database.
But how do we get back to a regular row when we need it ? We pivot it !
SELECT pkID, [Name],[Address],[Phone]
(select pkID, KeyCol, ValCol FROM unpivotSourceTable) uST
(MIN(ValCol) for KeyCol in ([Name],[Address],[Phone]) )as pST
pkID Name Address Phone
1 Bob Smith 1 Miller St Sydney 555-1234
2 John wayne 10 california Dr Hollywood +1-555-4567
3 Billy Bowden 4 Wainui road Wellington +64-21-3456789
Thursday, 13 October 2011
I am about to embark on a project with the Data services tool set. It seems like a great tool, but there are two things I am struggling to find at present:
- Any published books
- Good websites, blogs, forums etc.
If you have any good resources for me please leave a comment here for me.
Friday, 15 July 2011
Wednesday, 8 June 2011
Sunday, 27 February 2011
Saturday, 26 February 2011
Saturday, 19 February 2011
Friday, 18 February 2011
Saturday, 3 April 2010
Wednesday, 31 March 2010
An example of the type of problem we are facing is this: We get a file where the first character determines the format of the rest of the file. To get a complete transaction, you need to process 3 lines. For example we have (this is for a telco system):
Where [TAB] means a tab character.
So we get the account number and customer name on line 1, calling number on line 2, the called number, call type, duration and cost on line 3. There is a 1-N relationship from 1 to 2 and 2 to 3.
so rather than write the most complicated cursor ever we are using perl. The thing is i'm a bit rusty.
I'm interested to know:
a) What are some good books for re-learning perl (as opposed from starting from scratch)
b) Do you agree with the approach ? how would you tackle this problem ?
Sunday, 14 March 2010
My boss resigned last week largely due to the lack of focus management have been giving BI. Looks like a turbulent period ahead. Hopefully it gives management the wake up call (I believe) they need.
Thursday, 11 March 2010
What do I mean by this ? We have been quite a successful business intelligence team delivering to various areas of the business for the last couple of years so when management decided to put in a new multi-million dollar billing system we were brought in from the start to integrate this into out BI world.
And here is where things started to go wrong. Because we have more experience and skill in extracting data from our various systems, we were quickly roped into the data extraction, cleansing and migration tasks. Because the application vendor was too expensive in developing interfaces to our other systems and to external organisations, these became deliverables of the data warehouse. It has ended up that we are now responsible for 75% of technical tasks on the implementation of this new billing system.
Now we are receiving a lot of praise and thanks from management, but we no longer have capacity to cover the BI work we had planned for this financial year. Our people are getting frustrated and we are starting to lose staff.
The billing system still has another 3-6 months of work to complete (not counting any BAU tasks we may be left with at the end) and most of our team are out of patience.
I have tried to think of ways we can get out of this and back to our core area, but short of us screwing op royally (which we would never do intentionally) I can't see how we can convince management to let us return to our core area.
Any ideas ? Have you been in this situation and what did you do about it ?
Wednesday, 25 November 2009
Friday, 25 September 2009
But I have recently done a lot of reading of the subject of peak oil and now understand that that is just the tip of the iceberg. Our society is completely dependent of cheap power. Due to the continuing growth in both the developed and developing world, demand increases every year, but supply does not. On day soon - maybe tomorrow, maybe in 10 years, the amount of oil coming out of the ground will reduce. This will permanently change the dynamics of the world - we will find alternative fuels, but they won't be anywhere near as cheap, at least in the short term. Think of all the manufacturing that has been moved out of western countries - it only makes sense because it is cheap to transport to where the customers are. This applies to food also - it won't be economical to transfer fruit from South America to Europe for example. So now we come to this thing - the internet. The biggest cost (when you add the manufacturing, transport, and running costs) of this is power related. Will is continue to be feasible to ran suites like face-book of twitter when costs double of triple ?
I think there is a really danger that a lot of what we do will not last in the post oil world.. The millions of people sitting at PC's in office buildings may need to head out into the real world again and start growing and making things.. which might not be so bad after all !
Thursday, 17 September 2009
The first issue we became aware of is there are no 64 bit excel drivers available yet. Apparently they will arrive in Office 2010, which given we are using Office 2003 at the moment, won't arrive here for another 6 years. This has meant for about 1/2 our packages they will only work in 32 bit mode.
Next we found that BIDS only runs in 32 bit anyway, so if we addressed the above issue, we would have problems maintaining these packages anyway. This pretty much says to me that at least from an SSIS point of view, 64 bit SQL Server is not really for prime time.
Finally we had a lot of problem installing 32 bit ODBC drivers. They installed fine, but we then couldn't find them in the odbc administrator. It turns out their are 3 (maybe more ) obdc administrators in Win Svr 2K8 64bit. Solution: Using the odbcad32.exe located in %windir%\syswow64 .
I have seen the Microsoft adds that state the WOW starts now. Maybe they were referring to that directory...
NB: This process is ongoing, so I will update this post if we discover any more gotchas...
Monday, 6 July 2009
- Determine the Dimension at the first instance of the fact being recorded in the warehouse.
- Update the fact's dimension's to the version of the dimension that existed at the time of the fact being modified.
- Update to the Current version of the dimension at the time the fact is updated in the warehouse.
Option 2 to me is probably the best 'default' option in this case. It makes sense to me that if the fact is updated, it should reflect the dimensions at that point in time. For example if sale was made to the sales rep for region 1 (e.g. 'Bob') and Region 1 is later re-assigned to 'Fred', we don't necessarily want that sale updated to 'Fred' as he never made that sale. I'm sure there are cases though where this may not be the right approach.
Option 3 to me in most cases would give an incorrect view of the data, so therefore is not what we want either, however it may be that in the source system certain data is only updated periodically and that for that system the most recent version is deemed to be the 'correct' version.
I'm include to use Option 2 in most cases, but I would appreciate any feedback on this, as there may be other scenarios I haven't considered. Also, if you have come across any articles linked to this, please pass these on to me.
What are your thoughts ?
Monday, 27 April 2009
1: One of the “Big Three” (.NET, Java, PHP) -CHECK
2: Rich Internet Applications (RIAs) - FAIL
3: Web development -A BIT
4: Web services -A BIT
5: Soft skills -CHECK
6: One dynamic and/or functional programming language -CHECK
7: Agile methodologies -A BIT
8: Domain knowledge -CHECK
9: Development “hygiene” -CHECK
10: Mobile development -A BIT
So I give myself 7/10. Need to geek up a bit !
Thursday, 26 March 2009
Thursday, 12 March 2009
Monday, 19 January 2009
"And Gartner's fifth prediction is that by 2012 one-third of analytic applications applied to business processes will be delivered via coarse-grained mashups. IT shops are no longer focusing on grand visions such as SOA, which involve building composite applications out of fine-grained services or portals that merely display operational and analytical information next to each other, the analyst firm explained. Coarse-grained mashups, on the other hand, can be used to overlay analytical insights."
I interpret this, as tactical solutions will be successful, where as big bang solutions will fail. In my experience, that is almost always the case!
Friday, 17 October 2008
I found this link, to supposedly recession proof IT jobs. Thankfully, database administration is on the list. Lets hope its right !
Thursday, 9 October 2008
Thursday, 26 June 2008
So I updated my jruby environment to the latest (1.12 at the time of writing) and activerecord-jdbc-adapter to 0.81. As luck would have it some preliminary Sybase code had just been included in this release.
It didn't seem to work, however so I did some investigation. It was clashing with the MS-SQL driver (both can use the jtds driver, and it seems no one had considered the idea of one driver talking to different databases before). This started my interaction with the jruby community. People were extremely helpful and immediately made a change to how the database config worked to add a 'dialect' keyword so we could now use both sybase and MS-SQL with the jtds driver !
I was pretty pumped by now, so I set up the required files for running the simple unit tests, create a test sybase database (15.02 on XP) and started testing. Here is what I went though:
First run - 100% failure
Solution - the primary key definition was gibberish from sybase's point of view. By looking at what the MS-SQL driver did I was able to work out what was needed, added an overriding method and re-tested.
Second run - 70% Success !
This motivated me a lot so I addressed the next issues (drop index syntax) by adding another method and retested.
3rd Run - 99% success !
I was basically there. The last issue required no ruby code changes - just setting 'select into/bulk copy' on for the test database.
4th run - 100% success !
Ok, well there were lots more than 4 runs due to typos, database issues,etc. but only 4 that were relevant to getting the driver passing all the tests.
I have submitted a patch (article title links to the Jira ticket) and hopefully this will make it into an upcoming release. Is you don't want to wait, you should be able to apply the patch to activerecord-jdbc-adapter 0.82.
Not that I never have actually gotten to the rails app yet.. Hopefully soon. Now back to my day job :-(
Thursday, 22 May 2008
Wednesday, 12 March 2008
Hopefully I will be making more regular posts from now on...
Thursday, 15 November 2007
I decided, I'm learning Erlang. I've installed it (port install erlang - too easy) and I've order the pragmatic programmer's book. If you have any tips, pitfalls or general suggestions, drop me a note.
One downer is a quick search this morning came up with no jobs advertised for Erlang in Australia. I hope that changes soon. If you know of any Australian companies that use Erlang, please let me know !
Wednesday, 7 November 2007
Thursday, 25 October 2007
Friday, 19 October 2007
If you have any advice, please let me know !
Wednesday, 26 September 2007
What I'm trying to say is I've decided that in a couple of months I will move on from my current position and after a couple of months break I will be looking for a new position. If you think you might have anything for me starting Feb/March next year, let me know !
Wednesday, 15 August 2007
I bought a Mobits Adam A221 mini-tablet PC on Ebay. My plan was to get Linux working on it, but so far I have been unable to get either the WiFi or the Tablet/Pen interface working.
There seems to be next to no info on these on the web - if you have one and you have had any success could you let me know ?
Under windows the drivers it uses are:
Tablet - Fujitsu touchscreen driver. It is running on COM1
WiFi - Uses PRISM USB drivers.
Any ideas ? If I spend anymore time mucking around with this my wife will kill me !
Tuesday, 7 August 2007
One thing I have seen several times is that when companies can't find the person they want, they lower their expectations. They keep doing this until they can hire 'someone'.
This has several negative effects on the company, and not many positive ones:
- The person can't possibly fill the requirements the company had in the first place.
- Existing Employees will resent the fact that inferior people are being bought in, pushing more work to them without them receiving the benefits (which in a buyers market the new, inferior guy gets).
- 2 leads to more turnover, which can lead to more of these people being hired, which can cause things to snowball.
While I never appreciate being overworked, I really don't appreciate making up for people who aren't up to the job.
My view is if you can't find the right people you need to look at why you aren't appealing to those people. And if you are budget constrained don't throw your hands up in the air. Money is rarely the biggest factor in these things. You need to ask yourself what sort of people you need and what those people want. Things to think about include:
- Leave/working hours - not everyone is a 9-5*5*48 weeks a year kind of guy. The best person working 3 days a week or 9 months a year, is a far better asset than an inferior person doing 9-5 all year.
- work environment - If you insist your storage engineer wears a suit you will miss out on all the good storage engineers. If you insist your LAMP installation is deployed on a Windows, you will miss out on all the good Apache/Perl/etc. people.
- Telecommuting. If you trust people enough to let them work where they are most comfortable, they will do more work. If you don't trust your employees to work from home they won't trust you to reward them for the work they do.
So don't settle. Become the employer people want to work for !
Monday, 30 July 2007
Tuesday, 17 July 2007
Friday, 13 July 2007
How much time and money is wasted I wonder, because the decisions get made by the people higher up the chain as opposed to the people closest to the issues ?
Things never go well when people people aren't expected to 'eat their own lunch'.
Wednesday, 4 July 2007
- Please don't contact me about jobs in the finance industry. I already work in this industry and have lots of contacts here (unless you are offering 7 figure salaries ! )
- Please do contact me about jobs in other industries
- Please do contact me about jobs with flexible hours/days.
- Please do contact me if your project involves new and interesting technologies.
Monday, 2 July 2007
Here are what I believe some of the signs of this are:
- They consider Java a new and inovative technology
- They don't bother following industry news and developments
- They are resistant to the introduction of new technologies
- They worry about whether the open source software you propose will scale while they continue writing excel macros in vb script
- They think Linux is radical
- They are concerned about certifications from old school vendors
- They have been in the same job for more than 5 years
what do you think ? Am I being a bit harsh ?
Friday, 27 April 2007
( http://www.flickr.com/photos/7992966@N04/ ) if i've worked that flickr thing out properly.
Otherwise, I'll get back to you shortly.
Thursday, 29 March 2007
Personally, I have done this twice in my career and should have done it a 3rd time.
One thing that has always disappointed me is that companies seem to be willing to put up with bad managers and see their other staff leave. When the market is as good as it is now, you can easily see teams completely turnover in a year. You would think companies would learn !
So if your a manager complaining about the cost of new staff, ask yourself why the old staff left in the first place...
Friday, 23 March 2007
If you are one of those people that think struts and Hibernate are the best combination since the Captain and Tennille (cause I know you Struts people are into that shit :-) ), then you won't like it at all, because it takes you away from messing about with configurations files and makes you work on the problem at hand.
I really buy into the convention over configuration mantra for a couple of reasons:
- Opinions are like arseholes
- Most object-only guys can't (or don't bother to) get the database right, and activerecord encourages sorting out the design first
Wednesday, 21 March 2007
Whether the new benchmark (TPC-E) is any better, is the question. It is based on a financial scenario, which may work against Oracle (which has never been great at high volumes of small transactions which are typical of financial systems) a bit, but its unlikely to be much more real-world than the old one. Good companies will have to run their own benchmarks, like they always have.
Wednesday, 14 March 2007
"Technology is dominated by two types of people: those who understand what they do not manage, and those who manage what they do not understand."
Monday, 12 March 2007
Wednesday, 21 February 2007
Tuesday, 13 February 2007
I was reading an article on itnews.com.au about how CIO's are moving ahead of CFO's in some companies these days. I've never seen it myself , but I can't help but thinking this isn't a particularly good thing. In my career I have worked under probably a dozen CIO's (I won't count those when the organisation was a software house or an IT consultancy). Only one of those CIO's I would define as good at what they did (an ironically he was only an 'acting' CIO). If I was to characterise the CIO's I've worked under, the following are true of almost all of them:
They started off their careers in technical positions, but moved into the management side of IT as soon as they could.
They have never done any study in IT beyond undergraduate level (if your lucky), but always seem to have an MBA or other degree that looks good to non-IT people.
They don't understand current technology at all, but do understand vendor management.
They give the impression that they see being CIO as a stepping stone into becoming CEO (although i've never seen that happen).
Their direct reports are wannabe CIO's themselves, so all the above points are true of them also.
All together this paints a pretty ugly picture. These people are so out of touch, and it shows. The end result is you only ever get one of two messages from a CIO, ether:
"We need to cut costs and place better controls on out environment"
"We need to invest more and allow our people the freedom to innovate"
Thus it ends up being left to lower level people to run the organisation as best they can in between doing their real jobs.
Personally, I think the days of these CIO dinosaurs is slowly coming to an end. But it isn't us I.T. people that will bring them down.
Its the Marketing manager who asks why he can't install FireFox, only to realise the CIO has never heard of FireFox. Or the research analyst who it prohibited from using Python to develop their financial simulations because the CIO thinks Excel is what he should be using. These CIO's now know less about IT than your average person under 40, and once tech savvy non-IT people becomes CEO's these technically incompetent CIO's will disappear.
Good riddance to bad rubbish !
Monday, 12 February 2007
As I database consultant that has been working with Sybase and MS SQL Server for a long time I have been interested to see the rapid growth of MySQL. I decided to evaluate MySQL from the point of view of porting a database to it from Sybase/Ms SQL Server.
For those of you who don’t know, the two (Syb & MS) are still about 90% the same, at least from a Transact SQL point of view. They also both still come with that old friend – the pubs database (pubs2 & 3 on Sybase, pubs on MS SQL Server). Therefore what better to use for my sample database as pubs ! Note that I will not provide my converted script due to copyright issues, but if you are a Sybase of SQL Server user feel free to do the same. This is a very good test as pubs has been purpose build to try and use many of the different features available. Here is the results of my test starting from the point of my download of MySQL 5.0 alpha (the newest version available when I started this) :
1. Install MySQL, client tools & documentation.
2. RTFM for about 10 minutes.
3. set up my server using the winmysqladmin program. It gives me a working server ! Connect to the server: from a command prompt, type ‘mysql’ (no options).
I'm in a 'test' database. Follow a basic tutorial from the manual.
Learn a few basic commands like "show databases;" "select now();", & "select user();" - This seems pretty easy !
4. Decided to create a database. I'm going to create pubs2. Following the tutorial I find the syntax:
"create database pubs2;" Fails. Go back to RTFM.
5. Find out that there is a "test" user that I am connected as with no privileges. Find out I should be using the 'root' login. connect as root with no password:
mysql -uroot mysql. Follow recommendations to set a root password:
SET PASSWORD FOR root@localhost=PASSWORD('new_password');
Log out and login with the password:
mysql -uroot mysql -pnew_password
I'm in. Now try again:
create database pubs2; - Success !
Run "show databases;" - its there.
6. type "use pubs2"; - It works !
7. Cut and paste create table authors... from instpubs2. Get an error:
ERROR 1064 (42000): You have an error in your SQL syntax. Check the manual that
corresponds to your MySQL server version for the right syntax to use near 'id not null,
au_lname varchar(40) not null,
au_fname varchar(20) not null,
ph' at line 2
Sounds like a simple error, as id is a user define data type. Don't know how
to create those yet, so I'll replace it with the underlying type - varchar(11).
8. Try again - Success ! Now for Rest:
Publishers - ok
royshed - replaces tid with varchar(6), then ok.
sales - ok
-- Decided to work ou how to create udfs so I don't have to keep editing the scripts.
Spent about 1 hour earching the web and manuals. It doesn’t support them. Ill continue the way I was.
salesdetail - ok
titlesauthor - ok
titles - Error:
ERROR 1064 (42000): You have an error in your SQL syntax. Check the manual that
corresponds to your MySQL server version for the right syntax to use near 'money null,
advance money null,
total_sales int null,
notes varchar(200) null' at line 6
Thats right - no money datatype ! That wouldn't be a big deal if we had UDFs, but we don't. More re-writing ! (I’ll use decimal) and I am going to try replace image with the blob type. I don't know what the difference really is, but this is just a basic test. It works.
After getting through the tables. I get to constraints. Primary key constraints seem ok, but foreign keys can’t be created. After looking on the manual I find I need to create indexes before foreign keys. I do this, and can then progress. At this point however alarm bells are ringing about how it handles constraints. The manual is warning me that when using not null and default constraints that behaviour will not be as I would expect. For example if you try to insert a null into a not null column, MySQL will replace the null with a '0' or a '' (empty string) rather than rejecting the insert. Is sort of defeats the purpose and I would avoid using these features until they 'fix' them. The manual says you can recompile MySQL to change this, but doesn't make
it clear on what the new behaviour will be.
We need to skip some default clauses here. This requires editing the table creation scripts and manually placing the defaults on the relevant columns, or editing the inserts to include this. I have chosen to edit the data. Note I also had to replace
some occurrences of "" with a single ' as I am not sure how MySQL handles double quotes inside double quotes.
11. Indexes. MySQL doesn't seem to support the clustered/non clusters index types. I created the indexes without specifying the type and it works fine. The manual says that all MySQL indexes are b-trees with some compression.
12. Rules. Rules are not supported. I am going to ignore them as I think the only way to work around them would be to put the functionality in the app (not desirable) or in triggers.
13. Triggers. The manual says triggers are scheduled for 5.1. I can confirm they don't work in 5.0. I am having to push a lot of functionality back to the app (it theory – I’m not actually building an app).
14. Views. Not yet supported. Should be in 5.1 Derived tables (unnamed views) are supported, but this doesn’t help here.
15. Stored procedures. Apparently they are supported in 5.0 but I need to recompile the server. My C skills are probably not us to this, and it obviously isn't ready for prime time is it isn't in the server, so I'll forget them.
16. Grants. Grant all succeeded. Grant create table etc... didn’t. Give the number of things I have had to leave out, I don’t think I’ll worry about fixing this, but the manual suggest you can grant these things at the user level.
- Temporary tables have limited functionality and cannot be used in the same way as a real table. For example the following doesn’t work:
SELECT * FROM temporary_table, temporary_table AS t2;
- BCP like functionality is provided by the load data command. I only did some basic testing, but is seemed very fast (faster than bcp).
- The security model seems quite basic; groups and roles seem absent, so granting seems to need to be done on each user.
So what do I think of MySQL ?
This product has great potential. By the time version 5.1 becomes a stable production release it will be usable for a lot of applications. By the time 6.x comes out I would expect they have caught up in the important areas of data and domain integrity to the point where is will become a feasible option to companies out their. It wasn't that long ago that people tried to write off Apache & Linux. I can see MySQL being the next big thing in Open source, if it isn't already.
Having said that, as it stands now (5.0) I would not be willing to use MySQL for anything more critical than a reporting database. The lack of system wide rules, defaults and the problems with constraints mean that you would have to build all
of your referential and domain integrity into the app. That’s a DBA's worst nightmare!
It is worth noting here that there is an offshoot of MySQL called MaxDB (formally SAPDB). This has been developed by MySQL and SAP and has been developed to try and address these enterprise limitations. I have yet to look at it, but if I have time I will also try and conduct the same exercise against this.
People often ask me if Sybase will still be around in X years. The short answer is absolutely.
Firstly, this is my own personal opinion. As I used to work for Sybase I am somewhat biased, but I am presenting my reasons here and you can judge for yourself. Note also my figures are just to illustrate my point. If I did know the real figures (which I dont) I would not be allowed to use them anyway.
Since I first started working for Sybase around 7 years ago Sybase has grown in terms of RDBMS sales every year. "What ?" you say, I thought Sybase had lost market share. That really depends on what you are asking. The database industry (as has the whole software industry) has been growing every year for decades. There has been an explosion in the number of RDBMS systems deployed over the past 10 years and Sybase has grown with it. What you have probably heard (which is true) is that Sybase has lost market share as a percentage of market size. So when the world wide market for RDBMS systems was say, 500,000 (for arguments sake - I dont know the figures) Sybase had 20% of the market, or 100,000 customers. Due to Market growth the world wide market is now at least tens of millions (a conservative guess I would think). So if the worldwide market is now 20,000,000 (which does not seem as unreasonable guess) and Sybase has 3% of that market then Sybase has 600,000 customers - growth of 600% !
This of course refers to Enterprise RDBMS sales alone. Sybase sells over 100 other products (including several other databases) and some of those products are the most popular in their industry segment, for Example Adaptive Server Anywhere, which has around 70% of the mobile and embedded database market. Sybase is doing quite well really.
Update: Since I wrote this a few years ago, Sybase has acquired several companies, ASE 15 has been released and Sybase has continued to do fine.
From now on I'll replace it with this blog. Hopefully I actually keep this up to date !