ben robison

Everything I Need To Know About Life…

I learned in Isys 532 (not really, but this post is fishing for a grade. Dr. Liddle, this one is for you).

I’ll just throw out the warning up front that this is not a typical blog post. For my Information Architecture class, I’ve been instructed to condense a semester’s worth of learning into one blog post.

So my goal with this post is not to rehash all the research and work I’ve done throughout the semester, but to make a good parts version and wrap everything up. So here we go in no particular order. I wrote a random number generator to sort them for me. A real random-number generator, not a psuedo-random number generator.

Lesson #1: Ruby is not a toy. And Rails is not a buzzword.
Dr. Liddle, I can already see you shaking your head and smiling as you read this. To borrow a phrase, you’re thinking that all I’ve got is a hammer and everything looks like a nail.

While I admit that there may be a bit of truth in it, the fact is that I really don’t know that much about Ruby or Rails. But there are very smart people in the world that know just about everything that there is to know about it, and they’re working very hard to make it better.

It is not a solution for every problem. I’ve never seen a decent client-side GUI built on Ruby, but when you add Rails and throw it onto the web, it becomes a very powerful, very simple, and very stress-relieving application platform. It makes AJAX very simple. It teaches good practice. It has good separation of components. It has a very active friendly community around it. No it is not the solution to everything, but it is going to go places, and you should pay more attention to it instead of just smiling and shaking your head whenever I bring it up =)

Lesson #2: Your mileage may vary.
During this semester, I’ve spent countless hours on the internet, looking up different solutions to different problems, to put together programs and projects for technologies I’ve never even heard of before.

Even though I ran into the same problems and error messages that others had run into, their solutions did not always work for me. There are lots of folks that Google themselves onto my blog, and while I hope that my solutions work for them, that assumption requires a certain amount of naivety, and it’s not an assumption I’d make.

Sometimes an authoritative answer cannot be found, so we turn to the blogosphere. The so-called Web 2.0 movement has enabled millions of people to build themselves a spot on the internet and granted access to amazing information resources, but just keep in mind where this knowledge is coming from. Don’t be surprised when someone else’s solution does not work for you.

Lesson #3: More servers and computing power is not always the best answer.
Sometimes when you are presented with the same old problem, the same old answer is not the best one. I wrote a paper about Memcached this semester. It’s an innovative way to solve the ages-old problem of scalability.

Obviously when you get bigger, you’ve got to scale up, but the lesson taught by Memcached is that up is not the only direction to scale. Proper use of innovative technologies can enable you to scale out as well as up. And Memcached is free, so throwing more money on the fire isn’t always the best solution either.

I actually just ran across an article on Digg that provided instructions for surviving The Digg Effect. The article didn’t mention anything about buying new servers, but rather instructions for tuning your server, optimizing your DB, caching your results, and tweaking your PHP (read programming language of choice). That’s all free too.

Lesson #4: A Gentoo installation should not be undertaken in the middle of a busy semester.
This is pretty much self-explanatory.

Lesson #5: There is no perfect one-size-fits all solution.
There are a million ways to architect a system to support your application. Maybe Model Driven Architecture will fit your needs. Maybe it’s an extensible Service Oriented Architecture. Maybe it’s a simple Web Service that you publish for others to consume.

The point is that what works best for someone else, may not be your best option. Figure out what you really need, do your research, talk to people who know what they’re doing, and decide what works best for you.

Lesson #6: Don’t reinvent the wheel.
If you’re having some kind of pain with your infrastructure, chances are you’re not the first. Google is your friend. Or in an attempt to not be search-engine specific, the internet is your friend. There are a lot of smart and friendly people that can help you out.

Lesson #7: If you’re supporting a really really big, revenue-generating infrastructure, you’re going to want the right tools for the job.
This is somewhat related to the above point. We talked a lot about different architectures, applications, frameworks, and tools for solving your enterprise-level problems. Standards are developed for a reason. There are some industry standard tools for getting important tasks done. log4J and JMS are examples of accepted, widely used tools for enterprise logging and messaging. If a particular part of your IT infrastructure is not helping to generate a strategic advantage for your business, using tried-and-true solution is an excellent idea.

Lesson #8: Database optimization is not for the faint of heart.
Part of the reason it’s so difficult is because it’s different for every application and every person. It is hard, but it is not impossible.

Awhile ago I wrote about my thoughts for the best way to approach it. Benjamin Swanson, a good friend of mine, developed his own framework for performing the task. It’ll take time, but so do most other things that are worth doing.

Lesson #9: Putting in the extra time is worth the extra time.
I’ve found through personal experience that taking the time to do something right is worth the effort. Throughout my school career, I’ve built a reputation for being a go-to guy and getting things done because I’ve been willing to put in the time it takes to get things done. I find it rewarding when my peers ask me for help with something I’ve figured out.

But that is not why I do it. For me, putting in the time to learn something is its own reward.

*** Update ***

Lesson #10: Make your technology fit your business.
Perhaps this lesson is more powerful when stated in the longer form. Make your technology fit your business; don’t make your business fit your technology. Technology serves a purpose, but it is only a tool. When applied to the business world, it should solve a business need and more importantly, should make it easier for the business people to do their jobs. The MISM program at BYU emphasizes both the business and the technology, but in most cases, it’s the business that makes the money. The technology exists to help us do business better (well, obviously there are other uses for technology, but let’s keep things in context here).

MySQL Optimization

I’ve been doing some reading recently for a (long procrastinated) assignment at school. I’ve found some official looking materials as well as a recent Digg that led to some generally accepted industry expertise. I’ll give the links in a moment, but here’s the point of it all.

Low-hanging fruit
Don’t waste time trying to tune every query in your database. It can’t be done. Find the ones that take all the time and tune them. The slow queries log is a good place to start. My understanding is that you can set the parameters for the slow queries log in your my.cnf file.

log_slow_queries=/var/lib/mysql/slow­queries.log
long_query_time=2
log_long_format

EXPLAIN
Once you’ve found the queries that are taking a long time, you can use the EXPLAIN command (or the MySQL Query Browser graphical version) to see which parts of the query are taking the longest. It gives a lot of information about the query itself, but besides the times themselves the most useful information is related to the indexes. You can see how many records each portion has to crunch through, which indexes are available, and which indexes are being used. This leads to tip #3.

Index, Index, and….oh, Index.
Proper indexing is probably your best friend when it comes to increasing query performance. This has limited benefits for extremely dynamic databases with a high proportion of INSERTs and UPDATEs because the indexes will have to be rebuilt each time the table is changed, but where SELECTs are the bulk of the queries (most databases) indexing can contribute to huge increases in speed

More Info
http://dev.mysql.com/doc/refman/5.0/en/optimization.html
http://dev.mysql.com/doc/refman/5.0/en/explain.html
http://dev.mysql.com/tech-resources/presentations/presentation-oscon2000-20000719/index.html
http://jpipes.com/presentations/mysql_perf_tuning.pdf [pdf warning]
http://20bits.com/2007/04/10/10-tips-for-optimizing-mysql-queries-that-dont-suck/
http://immike.net/blog/2007/04/09/how-not-to-optimize-a-mysql-query/

Enterprise Java Tools

The Java Language presents many advantages for the enterprise-types.  My personal favorite language really shines on the web, but for the full architecture and support desired in enterprises, Java has a lot to offer (not the topic of this post).  Right now we’re going to look at 2 Java tools that have become commonplace in enterprise development: log4j and JMS.

log4j
log4j (non-capitalized on purpose) is an open-source project developed under the Apache Jakarta banner.  It was originally developed by Ceki Gülcü. It is built to be the best Java logger around, and the industry agrees that it is.  It’s used for debugging and logging and can provide detailed context for application failures.

It is built as a package that you can drop into your application and start making calls to the logger object.  log4j may provide the best or only solution for debugging (this is often case in distributed applications).

log4j enables logging at run-time without modifying the application’s compiled binary.  This means that you can enable debugging without suffering the heavy performance loss that you might expect by bloating your binaries with extra code.

Perhaps one of the most useful features in log4j is the idea of hierarchy and inheritance.  As you make calls to the logger object and log different kinds of events, you can have different outputs that listen to the logs and pass them on when certain criteria are met.  It’s sort of a subscriber provider model for things.  Potential subscribers would be loggers outputting information to the console, files, databses, SMTP servers, GUI components, etc.  These subscribers are called appenders.

This means that you could potentially have everything logged to a standard file, important things put into a different file, database, etc., and have critical errors emailed to someone that can fix them.  The only thing you’d need in your application code would be the one call to the logger object and everything else happens as the different loggers subscribe to the different events.  All the subscriptions to events are handled by external configuration files, so you don’t have to modify your application code in order to change the behavior of the loggers.

JMS
The Java Messaging Service is another logging solution.  My research on this topic introduced the phrase “asynchronous logging” for the first time although it sounds like log4j works on the same principle.  Reading about JMS also used the publisher/subscriber metaphor which I unfortunately already used, but these two facts should illustrate some of the similarities.

JMS actually supports to different types of logging.  The queuing model has a producer and only one consumer.  They are independent, so the consumer does not have to be running when the producer produces, and the producer does not have to be running when the consumer consumer.  This is a higher level of abstraction than log4j uses (according to my understanding).

The Publish/Subscribe model published messages in “topics.”  Zero or more subscribers can subscribe to a particular topic.  In this model the subscribers and the publisher are not even aware of the other’s existence.

Some enterprising individual has even built an Appender for log4j that plugs the JMS into log4j.  O’Reilly has also published a book on the JMS which in my book means it’s significantly more complex.  From my readings, log4j is also further abstracted out of the application than JMS.  It sounds like in order to change what your JMS publisher is doing, you’ll have to change code in your application and recompile.

Model Driven Architecture [part 3]

This will be a short post, but I thought it would be interesting to look at one specific offering of an MDA product to see what they can do.  OptimalJ is made by Compuware and provides MDA programming for those of use that live in a J2EE world.

To start things off, this PDF file is one of their marketing releases, but as we look through it, we can see the foundations of what makes an MDA product.

OptimalJ consists of three basic components.  (1) Metamodels create separation between application design, infrastructure, and code.  (2) Technology patterns make model-to-model transformation possible. (3) Finally, implementation patterns allow for the model-to-code transformation.

Metamodels: At the high level, you need to separate the different aspects of your applications so that you can address the concerns of each individual piece before you bring it all together to work out those issues.  Since we’ve started breaking things down into threes (3 MDA posts and 3 OptimalJ components) we’ll do one final three-part breakdown.  There are three kinds of metamodels.  The domain model, the application model, and the code model.

Proper usage of these metamodels formally defines the kinds of data needed by the application,  what technology the application will be using, and specific code pieces that will be used in certain situations.  This last part is a bit confusing, but the OptimalJ product allows for editing of templates (of a sort) that will later be used during the translation of model-to-code.

Technology Patterns: above we mentioned that this provides model-to-model transformation.  Specifically, this provides for the transformation of the domain model into the application model.  So starting from the kind of data that we want, we now translate that into the architecture of our application.  What kind of application is this?  Web-based, client-server, etc.

Implementation Patterns:   this transforms the application model into the code model.  This is where actual code is produced.

OptimalJ provides graphical editors for all these different pieces, but breaking it down this way, we can begin to see what OptimalJ can do.  We start up front with the need for an application.  After defining what data we need in the data model, we can generate an application model that provides an overview of the application itself.  Further transformation into the code model gives an application that can be compiled and run.

If we change the way the data should be stored, we can regenerate the application model and the code model and cut out all the in-betweens of recoding to match our new data structure.  All this auto-generation even includes SQL scripts for datbase creation if you’re willing to spring for the higher-end versions.  Eclipse support and Netbeans support virtually guarantees that you’ll be in a familiar environment.  Is there anyone in the world that still writes Java outside of those IDEs?

Is Model Driven Architecture Right For Me?

Is represents Part 2 in the MDA series. In part one we talked about what exactly MDA is and what it can do. In this section, I wanted to talk a little bit more about how if fits in different places. This isn’t meant to be a comprehensive source of information (it’s just a blog), but I would like to discuss some of the things I would look at before making the MDA leap.

How much software development does your firm do? Not all technology firms are involved in software development, and if yours is one of the ones that doesn’t do some hardcore development, MDA is probably not your thing.

For development firms, there are major benefits to be realized under certain circumstances. Richard Blais from Locus Systems Inc. believes that

…software development is not necessarily an art form. It requires discipline, it requires a process…

He talks about the benefits of using OptimalJ in a short video clip. I think it does a good job of illustrating benefits for large development companies. Some of the highlights are:

• Solid code generation, good process within the code
• Reuse of code & effort (4-5 of 10 things that have to be done for new applications are already done)
• Shortened SDLC (55% decrease in time to market)
• Empowers software architects and programmers
• One source of a competitive advantage
• 44% revenue growth powered by OptimalJ

What if you’re not running a big development firm, but you’re still in the ball game somewhere. MDA could be a good fit for your business, but it might not be. How do you know? Maybe it’s a good fit for certain projects and not for others.

Let’s take my hypothetical web development business that helps local businesses get themselves up on the web. Some of them would want to actually sell products online, but others just want to start a web presence. Some want to handle online scheduling and others just want their information available. Is MDA a good fit?

These are some of the things I would look at to decide if MDA is a good fit for my little projects.

Complexity: If the business is very complex, or particularly when the data model is complex, MDA begins to become a stronger candidate. In my mind, MDA can really shine through when you’ve got complex rules on top of complex data structures. Even if you’re only using it for the first round implementation, a lot of the complexity can be reduced by an MDA product.

Size: Again, the bigger the project, the more potential that MDA can knock out a significant amount of the work up front.

Flexibility: This is closely related to the next point, but if there is a lot of flexibility required, perhaps MDA is not the greatest candidate.

Ambiguity: If the requirements are ambiguous or the client is figuring things out as you go along. MDA won’t be much help here. Without a fairly completed definition up front, generating code from a model becomes problematic and difficult. If you run into this problem very often, you might have a little business problem in your own business besides a difficult client relationship to manage.

Time line: I suppose this is related to the first two points as well, but on a tight time line, an MDA product of one sort or another can really help you front load the project and more complex routines taken care of at the beginning. If you use MDA often and consistently, you might find yourself in the enviable position of Locus and have several components of code that you can reuse from project to project. That doesn’t mean that MDA is the only (or even the best) way to reuse code, but the required levels of formality and process make it a much greater likelihood.

Existing code base: If you’ve got an existing code base, in my mind I would tend to swing away from MDA. Partially (or poorly) implemented projects tend to have less documentation and formality. In my experience, that doesn’t give you enough information up front to go the MDA route.

I think I’ll wrap up tonight’s post at this point. In the last and final installment in my MDA series, we’ll take a look at OptimalJ more in depth to see some of the specific benefits it (and MDA in general) can offer.