When I was growing up, while on trips with the family, we'd always ask my dad if we were there yet. For some reason, we needed to know how much longer we'd be in the vehicle. My dad's answer was always the same. A half hour. This seemed to satisfy us for a while. When we repeated the request, the answer was the same, regardless of the actual time.
Obviously we stopped believing the answer with time. But that didn't prevent us from asking and more importantly, from wanting to know. My own children ask similar questions. My son likes to know what we are doing, or watching, and so on. We humans like to know that things have a finite length of existence. We like to plan things. Start them. Know when they will end or when they will be over.
Recall when you would copy a file in older versions of Windows, the flying file icon and the guessed completion time. It aggravated me to find out that this was just a guess, and was often wrong. Why couldn't it tell me the correct duration? Why couldn't I be privy to this knowledge. It would help me to plan what I would do next, based on how long this action could take.
Where am I going with this? The other day at work, we were performing some tasks. We kicked off the task. As soon as it was running, we wanted to know how long it would take to complete. Sometimes we can lean on scripts and processes that we have previously written. When we do not have canned scripts or tools ready, we often panic. We will try to invent something on the spot. In a few occasions I have asked the twitterverse for assistance, and received help there. Other coworkers have provided help as well. But each time, I wanted to know much sooner than I was able too, when it would be done. What I should have done was taken the time before hand, to create the needed script or tool or means to determine progress, prior to kicking off the process. This is not always an option. Its a good idea though.
Once the process has completed, I want so desperately to spend time writing these scripts or tools. If I can, I will, and save them for the next time. Keeping these tools handy will better allow me to answer these questions next time. Take the time now to better prepare these tools. Do it when your head is clear, and you can write them without the pressure of a running process. Share them with others, coworkers and the SQL community.
Ideas
backups, native and 3rd party
bcp or bulk insert processes
dts/ssis packages and its steps
reindexing scripts
attaching/restoring database
These are the ones that appear on the tip of my tongue. You will have others that ring a bell for you. Identify them now. Come up with the tools to answer that burning question, 'Are we there yet?'
Thursday, April 23, 2009
Wednesday, April 22, 2009
Release Management - how do you do it?
I've been in different IT shops with many different ways of releasing code to the database. Some have been very loose in their security and processes. Simply pushing code thru a sql tool straight into production, without even testing it in other environments. Sometimes it was written by the DBA or by a developer.
Other IT shops have had much more strict release management processes. Change control boards meet to discuss the changes and approve or disapprove of them. Actions occur based on these decisions.
Still other IT shops have had many layers of systems in the which sql code was released, tested, approved. Different groups pushing code to different servers along the way. Production pushes being the most complex, with a lot of people involved on conference calls to ensure proper execution of planned steps.
I have heard of the perfect world, where everything had been taken care of for us, and at the push of a button, code is released in a controlled fashion, with no problems whatsoever. none of us live there. So lets talk about the world we live in.
For production releases, we have instituted a process that forces as little human interactions with the code as possible. A tool had been created that executes the sql code. This code is stored in a source control system. There is a single document that describes the actual files to be pushed, with filenames. The files are all listed out in this file, within specific groupings and the tool can parse this data and manage it for us. We simply load a production file, and the tool pulls in all the needed files. We can select specific ones to be released, or release them all at once. We simply indicate the server to be released too, and hit a button.
This sounds easy, but it still has some complexities. With certain types of code that will be pushed, we have to do it in a specific fashion. For example, data that needs to be released should only go to certain servers. Pushing ad-hoc sql is not allowed. We've instituted a process for data changes to be included in a stored procedure. This proc is released, executed and dropped. I know what you are thinking, isn't this the same as a human executing the sql? Remember, this is one of the goals, removing the human interactions. The old way we'd do it was to execute the code in a sql tool. But we could select the wrong portion of code, or miss typed the proc name we released, or even put in the wrong parameters.
So, putting the SQL into a tool that performs the action removes that human interactions, the possibility of the DBA miss selecting portions of code. Putting the SQL code into a proc, releasing this to the appropriate database, executing it and dropping it, all from the release tool, makes it even more removed from the human interactions. By forcing this release to be performed and tested in multiple environments prior to production, we can be ensured that it should succeed, in production.
So far, so good. We've been able to successfully release code into all environments in the fashion with few goofups. Where previously we had instances that we executed code in a tool and caused issues. Performing a release is stressful enough. Why introduce more layers of problems with possible code issues.
Well, like it or not, thats the process we are supporting currently. There are parts i like about it, and parts i would like to alter, enhance, and so on. What I'd like to know is how others do their code pushes, mainly to production. Specifically, how you release data changes. There are few permutations to releasing a schema change, new objects, or security. Those are fairly cut and dried. But how we push data changes, I;d like to know how you do it.
Comment on your process, what you like, what you dislike, how you would do it in a perfect world. Lets all learn from each other.
Other IT shops have had much more strict release management processes. Change control boards meet to discuss the changes and approve or disapprove of them. Actions occur based on these decisions.
Still other IT shops have had many layers of systems in the which sql code was released, tested, approved. Different groups pushing code to different servers along the way. Production pushes being the most complex, with a lot of people involved on conference calls to ensure proper execution of planned steps.
I have heard of the perfect world, where everything had been taken care of for us, and at the push of a button, code is released in a controlled fashion, with no problems whatsoever. none of us live there. So lets talk about the world we live in.
For production releases, we have instituted a process that forces as little human interactions with the code as possible. A tool had been created that executes the sql code. This code is stored in a source control system. There is a single document that describes the actual files to be pushed, with filenames. The files are all listed out in this file, within specific groupings and the tool can parse this data and manage it for us. We simply load a production file, and the tool pulls in all the needed files. We can select specific ones to be released, or release them all at once. We simply indicate the server to be released too, and hit a button.
This sounds easy, but it still has some complexities. With certain types of code that will be pushed, we have to do it in a specific fashion. For example, data that needs to be released should only go to certain servers. Pushing ad-hoc sql is not allowed. We've instituted a process for data changes to be included in a stored procedure. This proc is released, executed and dropped. I know what you are thinking, isn't this the same as a human executing the sql? Remember, this is one of the goals, removing the human interactions. The old way we'd do it was to execute the code in a sql tool. But we could select the wrong portion of code, or miss typed the proc name we released, or even put in the wrong parameters.
So, putting the SQL into a tool that performs the action removes that human interactions, the possibility of the DBA miss selecting portions of code. Putting the SQL code into a proc, releasing this to the appropriate database, executing it and dropping it, all from the release tool, makes it even more removed from the human interactions. By forcing this release to be performed and tested in multiple environments prior to production, we can be ensured that it should succeed, in production.
So far, so good. We've been able to successfully release code into all environments in the fashion with few goofups. Where previously we had instances that we executed code in a tool and caused issues. Performing a release is stressful enough. Why introduce more layers of problems with possible code issues.
Well, like it or not, thats the process we are supporting currently. There are parts i like about it, and parts i would like to alter, enhance, and so on. What I'd like to know is how others do their code pushes, mainly to production. Specifically, how you release data changes. There are few permutations to releasing a schema change, new objects, or security. Those are fairly cut and dried. But how we push data changes, I;d like to know how you do it.
Comment on your process, what you like, what you dislike, how you would do it in a perfect world. Lets all learn from each other.
Wednesday, April 15, 2009
SQL Quiz : Who has been a great leader in your career and what made them a great leader?
I was tagged by Chris Shaw in his latest SQL Quiz.
As Chris mentioned, great leaders and great managers are not always embodied in the same person. I take this a bit further, and a person may even have flashes of being a great leader or manager or both, and not sustain it. It is difficult to sustain both, but it can be accomplished.
1 bad example was a boss I had recently. His idea was that his people needed training. They needed time set aside for said training. It was understood that you would find the time and get the training you could. This was rather vague and open ended. I completely agree that a company should allow its people receive training, in many different forms, and it should be allowed. The downfall was that no follow through occurred. It would have been better if there was a structure, suggestions, reporting, and so on, on our progress and training. But now. So, that would be an anti-example. But it was headed in the correct direction.
My suggestion here is to create a plan of action to follow through, if this is a goal that you as a manager have. Make it happen. Make it make a difference in the lives of your employees. Not just a 'talk about'.
Let me now give you a good example of a manager and leader. As DBA's, we often work strange hours. This lifestyle is difficult to get used too. I have been in many jobs in IT where I was expected to be in my desk between 8 and 5. If I was not at my desk, it was assumed I was not working. We know that this is not the case, most of the time. This habit was ingrained into me in such a fashion that I had a hard time with a job that required odd hours of work. Some weeks, I would end up working my normal block of 40 hours, only to find that I needed to work 10-20 hours after hours. The expectation that I had was that my boss would somehow 'know' what was going on, and reward me with comp time. As this perceived reward never materialized, I grew increasingly frustrated. This soon became apparent, to my manager. In the best way possible, I would describe my frustrations of how much I was working, how much extra, all the personal events that I was missing, and so on. His response took me off guard. He asked me why I had allowed this to occur. My response to this was, 'what a jerk!' He should have seen how hard I was working and given me that 'reward'. We went on to talk, and I learned some great lessons. He didn't feel that it was his job to ensure that I was not being overworked. He had laid the groundwork of this, set the policies, and expected us to comply. It was within me and under my control to comply and thus, live my work life appropriately.
Granted that this interaction seems short and enlightened here, it was more painful and grueling to work through in my head. The point was he was not only teaching me how to fish, but dumping me on the banks of a river, and letting me do it on my own.
He managed his employees expectations of a work/personal life balance. He lead with example. He expected us to simply comply and have a better life for it.
It has taken some practice, but its a much better life on this side of the fence because of it.
I guess my point here was that a great leader and manager can coexist. They have to see the problems, come up with solutions, continually manage the people until they became habitualized with the new way of doing it, then let them do it on their own. Then he would move onto the next task, and repeat. It has been an interesting journey to watch and travel. Work life balance has improved, moral has improved, quality of work has improved. And its not over. Its a continual battle.
I am tagging the following people.
Kendal Van Dyke (who was tagged by BrentO already)
Pat Wright
Tim Ford
As Chris mentioned, great leaders and great managers are not always embodied in the same person. I take this a bit further, and a person may even have flashes of being a great leader or manager or both, and not sustain it. It is difficult to sustain both, but it can be accomplished.
1 bad example was a boss I had recently. His idea was that his people needed training. They needed time set aside for said training. It was understood that you would find the time and get the training you could. This was rather vague and open ended. I completely agree that a company should allow its people receive training, in many different forms, and it should be allowed. The downfall was that no follow through occurred. It would have been better if there was a structure, suggestions, reporting, and so on, on our progress and training. But now. So, that would be an anti-example. But it was headed in the correct direction.
My suggestion here is to create a plan of action to follow through, if this is a goal that you as a manager have. Make it happen. Make it make a difference in the lives of your employees. Not just a 'talk about'.
Let me now give you a good example of a manager and leader. As DBA's, we often work strange hours. This lifestyle is difficult to get used too. I have been in many jobs in IT where I was expected to be in my desk between 8 and 5. If I was not at my desk, it was assumed I was not working. We know that this is not the case, most of the time. This habit was ingrained into me in such a fashion that I had a hard time with a job that required odd hours of work. Some weeks, I would end up working my normal block of 40 hours, only to find that I needed to work 10-20 hours after hours. The expectation that I had was that my boss would somehow 'know' what was going on, and reward me with comp time. As this perceived reward never materialized, I grew increasingly frustrated. This soon became apparent, to my manager. In the best way possible, I would describe my frustrations of how much I was working, how much extra, all the personal events that I was missing, and so on. His response took me off guard. He asked me why I had allowed this to occur. My response to this was, 'what a jerk!' He should have seen how hard I was working and given me that 'reward'. We went on to talk, and I learned some great lessons. He didn't feel that it was his job to ensure that I was not being overworked. He had laid the groundwork of this, set the policies, and expected us to comply. It was within me and under my control to comply and thus, live my work life appropriately.
Granted that this interaction seems short and enlightened here, it was more painful and grueling to work through in my head. The point was he was not only teaching me how to fish, but dumping me on the banks of a river, and letting me do it on my own.
He managed his employees expectations of a work/personal life balance. He lead with example. He expected us to simply comply and have a better life for it.
It has taken some practice, but its a much better life on this side of the fence because of it.
I guess my point here was that a great leader and manager can coexist. They have to see the problems, come up with solutions, continually manage the people until they became habitualized with the new way of doing it, then let them do it on their own. Then he would move onto the next task, and repeat. It has been an interesting journey to watch and travel. Work life balance has improved, moral has improved, quality of work has improved. And its not over. Its a continual battle.
I am tagging the following people.
Kendal Van Dyke (who was tagged by BrentO already)
Pat Wright
Tim Ford
Tuesday, April 07, 2009
SSWUG Virtual Conference
Yesterday, I had the distinct pleasure and privilege to goto Tucson Arizona and present on some topics for their Virtual Conference occurring in April 22, 23 and 24. Back in December, Chris Shaw contacted me and asked me if I would be interested in speaking for their organizations virtual conference. I was humbled and surprised to be requested. I prepped a bunch of presentation ideas, abstracts, and submitted them to Chris. I had 7 accepted, 4 for the Virtual Conference, and the others for the ITUplink. So, for the past few months, I have been preparing presentations, practicing, adding demos, and trying to perfect them. It does seem that each time I look at a slide deck, I find a way to alter it, perfect it a bit more. At some point, I have too stop editing them, and simply give the presentation.
I had the chance to practice a couple of these on my User Group, and glad that I did. The more practice, the better. I worked out some kinks, and got further ready to present to the larger audience. Even though, while doing it, there is no audience. You have to imagine it. They will be there, but not when you are recording it. Practicing in front of an audience, large or small, really helps. But presenting to no one is useful too. I've presented a few times to a phone. Or a live meeting with no one talking to you or giving feedback. Each type of presentation is unique and rewarding in its own way. But each is very different.
Presenting for the SSWUG organization is in its own sphere, similar to the other ways described, but unique as well. Having to look into a camera, well, thats something that is not normal for the DBA. If anything, I am usually behind the camera in my personal life, taking pictures of my family. I rarely end up in front of the camera. And especially not in front of nice cameras like the ones at Bits on the Wire. And 3 of them. One on the right side, one on the left, and in the far left, another one that is stuck in the corner, and apparently black & white. While you are presenting, and trying to not lean on the table in front of you, and remember where you are supposed to stand, and the fact that you have no audience to give you feedback, you also have to remember which camera to look into. If you forget, the fine folks in the recording booth will flash a light on one of the cameras to remind you which one you should be looking into. In between the two color cameras, there is a screen with a clock. It lets you know how much time you have used. I would end up looking at it on occasion, and out of the corner of my eye, see a flashing blue light reminding me the camera to look at.
I should backup and describe my experience more fully, though not too detailed. Just the highlights. I chose to film on a Monday, so that I didnt eat into too much work time off. I had to fly out Sunday night to ensure that I reached Tucson to film on Monday. I got in late Sunday, went down the stairs of the airport, and was greated by a gentleman that gave me a ride to the hotel. I had secretly hoped I'd see my name on a paper, cause I've never had that happen to me yet. But a paper with Bits on the Wire was close enough. After getting into the hotel, I had some time to chill and get to sleep. Not terribly eventful. The next morning, I had the entire morning to do nothing, as I was to report to the studio around 1pm. Waking up late seems lie a luxury these days, and I took full advantage of that Monday morning. Waking up late, wondering downstairs to get a free breakfast, with no one telling me what to do, seemed like the most glorious of vacations. After eating, I returned to my hotel room and prepped my presentations. As I have mentioned earlier, I kept making more and more changes.
Before I knew it, it was time to shower, grab some food and then meet my ride to the offices. Once there, I was able to meet Stephen Wynkoop and the rest of the crew. Stephen even sat with me and interviewed me for a bit, making me feel like a celebrity he was excited to finally meet, just as I felt it was to meet him, finally. We had a great little discussion, followed by me jumping right into my presentations. After a quick practice start to ensure I knew which camera to look at (I would later forget), we jumped into the real presentations. The first one that I recorded was the most prepared and ready. This was planned by me, so that I could get an easy one under my belt, before struggling a bit on some parts of the other ones. One after another we cranked them out. In between I would grab my BlackBerry, send a quick tweet, and check for missed calls or emails. Just a quick glance, then back to the sessions and onto the next one. Before I knew it, I had finished 4 presentations, taking close to 5 hours of time. Whew. It was pretty intense, pretty enjoyable, pretty wild.
Once it was all over, a euphoria overcame me. I had one of those natural highs for a bit. I tried to sit back at the hotel, but became bored with TV. Hungry. Yeah, thats it. Venturing out into the outside world, I soon found food. At sometime afterwards, I started to come down from my natural high. Work crept into my world at this time, and I spent a few hours connected to work via VPN until it was bed time. Rising early the next day, I went to the airport earlier than i needed too, and awaited my flight. Once on the ground again in SLC, it was back to normal and back to work.
All the while, before, during and after, I tweeted my journey via Ping.fm. Posting went to Facebook as well as LinkedIn and Twitter. People in all those worlds were able to follow my progress and adventure, adding to my trip with their insights and suggestions. I love being alone, yet connected to so many people, all at the same time.
I hope that my sessions prove to be helpful for people, once they are released in the virtual conference. In the meantime, take a peek at what has been posted as a preview of one of my sessions.
I have a discount code for the SSWUG Virtual Conference. You can register for the conference here and make sure you use my code 'SPVTBESP09'.
More information here : http://tjaybelt.blogspot.com/2009/02/sswug-vip-code.html
I had the chance to practice a couple of these on my User Group, and glad that I did. The more practice, the better. I worked out some kinks, and got further ready to present to the larger audience. Even though, while doing it, there is no audience. You have to imagine it. They will be there, but not when you are recording it. Practicing in front of an audience, large or small, really helps. But presenting to no one is useful too. I've presented a few times to a phone. Or a live meeting with no one talking to you or giving feedback. Each type of presentation is unique and rewarding in its own way. But each is very different.
Presenting for the SSWUG organization is in its own sphere, similar to the other ways described, but unique as well. Having to look into a camera, well, thats something that is not normal for the DBA. If anything, I am usually behind the camera in my personal life, taking pictures of my family. I rarely end up in front of the camera. And especially not in front of nice cameras like the ones at Bits on the Wire. And 3 of them. One on the right side, one on the left, and in the far left, another one that is stuck in the corner, and apparently black & white. While you are presenting, and trying to not lean on the table in front of you, and remember where you are supposed to stand, and the fact that you have no audience to give you feedback, you also have to remember which camera to look into. If you forget, the fine folks in the recording booth will flash a light on one of the cameras to remind you which one you should be looking into. In between the two color cameras, there is a screen with a clock. It lets you know how much time you have used. I would end up looking at it on occasion, and out of the corner of my eye, see a flashing blue light reminding me the camera to look at.
I should backup and describe my experience more fully, though not too detailed. Just the highlights. I chose to film on a Monday, so that I didnt eat into too much work time off. I had to fly out Sunday night to ensure that I reached Tucson to film on Monday. I got in late Sunday, went down the stairs of the airport, and was greated by a gentleman that gave me a ride to the hotel. I had secretly hoped I'd see my name on a paper, cause I've never had that happen to me yet. But a paper with Bits on the Wire was close enough. After getting into the hotel, I had some time to chill and get to sleep. Not terribly eventful. The next morning, I had the entire morning to do nothing, as I was to report to the studio around 1pm. Waking up late seems lie a luxury these days, and I took full advantage of that Monday morning. Waking up late, wondering downstairs to get a free breakfast, with no one telling me what to do, seemed like the most glorious of vacations. After eating, I returned to my hotel room and prepped my presentations. As I have mentioned earlier, I kept making more and more changes.
Before I knew it, it was time to shower, grab some food and then meet my ride to the offices. Once there, I was able to meet Stephen Wynkoop and the rest of the crew. Stephen even sat with me and interviewed me for a bit, making me feel like a celebrity he was excited to finally meet, just as I felt it was to meet him, finally. We had a great little discussion, followed by me jumping right into my presentations. After a quick practice start to ensure I knew which camera to look at (I would later forget), we jumped into the real presentations. The first one that I recorded was the most prepared and ready. This was planned by me, so that I could get an easy one under my belt, before struggling a bit on some parts of the other ones. One after another we cranked them out. In between I would grab my BlackBerry, send a quick tweet, and check for missed calls or emails. Just a quick glance, then back to the sessions and onto the next one. Before I knew it, I had finished 4 presentations, taking close to 5 hours of time. Whew. It was pretty intense, pretty enjoyable, pretty wild.
Once it was all over, a euphoria overcame me. I had one of those natural highs for a bit. I tried to sit back at the hotel, but became bored with TV. Hungry. Yeah, thats it. Venturing out into the outside world, I soon found food. At sometime afterwards, I started to come down from my natural high. Work crept into my world at this time, and I spent a few hours connected to work via VPN until it was bed time. Rising early the next day, I went to the airport earlier than i needed too, and awaited my flight. Once on the ground again in SLC, it was back to normal and back to work.
All the while, before, during and after, I tweeted my journey via Ping.fm. Posting went to Facebook as well as LinkedIn and Twitter. People in all those worlds were able to follow my progress and adventure, adding to my trip with their insights and suggestions. I love being alone, yet connected to so many people, all at the same time.
I hope that my sessions prove to be helpful for people, once they are released in the virtual conference. In the meantime, take a peek at what has been posted as a preview of one of my sessions.
I have a discount code for the SSWUG Virtual Conference. You can register for the conference here and make sure you use my code 'SPVTBESP09'.
More information here : http://tjaybelt.blogspot.com/2009/02/sswug-vip-code.html
Sunday, April 05, 2009
Wednesday, April 01, 2009
SQLPASS Auditing & Compliance SIG LiveMeeting
Today, the Auditing & Compliance SIG had our first live meeting event. We held a presentation about Sarbanes-Oxley and the DBA.
Many different topics were presented, with possible solutions provided.
We discussed ways to remedaite the following topics, and how to provide auditable documentation for auditors.
Change Management
Division of duties
Backup and Recovery
Restoration
Data Structure Changes
Back End Database User Access
Database Authentication & Access
Job Monitoring
Data Monitoring Controls
Data Transfer
Physical Access
I felt like it went well, and hopefully helped folks out with their concerns and needs with this topic. A few people asked for the slides or the recording. I hope that it got recorded, but in the short term i have posted the slides here.
When I find out about the recording, I'll post it somewhere we can get to it.
I hope that this is the first of many events that will help us DBA's with Auditing & Compliance.
Many different topics were presented, with possible solutions provided.
We discussed ways to remedaite the following topics, and how to provide auditable documentation for auditors.
Change Management
Division of duties
Backup and Recovery
Restoration
Data Structure Changes
Back End Database User Access
Database Authentication & Access
Job Monitoring
Data Monitoring Controls
Data Transfer
Physical Access
I felt like it went well, and hopefully helped folks out with their concerns and needs with this topic. A few people asked for the slides or the recording. I hope that it got recorded, but in the short term i have posted the slides here.
When I find out about the recording, I'll post it somewhere we can get to it.
I hope that this is the first of many events that will help us DBA's with Auditing & Compliance.
Subscribe to:
Posts (Atom)
