Data Encryption paper published on SQLServerCentral

The focus of this paper is on the protection of PII data stored within a database using encryption technologies.  Read it on SQLServerCentral.

If you don’t have a SQLServerCentral account then read it here.

There’s a dark side to everything.

The Big Data menace continues to grow as reported by Ryan Gallagher of The Guardian: “A multinational security firm has secretly developed software capable of tracking people’s movements and predicting future behaviour by mining data from social networking websites.”  Raytheon’s system named RIOT is fed from individuals’ voluntary posts to social media sites; during a demonstration Raytheon  was able to identify their “example target. With information gathered from social networks, Riot quickly reveals Nick frequently visits Washington Nationals Park, where on one occasion he snapped a photograph of himself posing with a blonde haired woman. We know where Nick’s going, we know what Nick looks like now we want to try to predict where he may be in the future.”

An amazing, if not Orwellian, example of the use of predictive analytics.

From my soapbox:

Everything you share online on any website may be used against you.

Big Data: Promise and Menace

I recently read that big data is “a solution looking for a problem” (Gartner). I’ve also read that many companies are monetizing their Big Data: “An overwhelming majority of companies (73 percent) have already leveraged data to increase revenue. Of those companies that have already increased revenue, 57 percent used data to increase an existing revenue stream.” (Global Survey: Is Big Data Producing Big Returns?) Certainly, the Big Data excitement is building as evidenced by the following chart:

As a DBA I see Big Data as a new opportunity. There appears to be real work to be done with Big Data in the medical, pharmacology, and other scientific fields as well as with customer-facing enterprises: new discoveries and services await – that is the promise. Unfortunately, Big Data is also invading our privacy providing data aggregators, data vendors, and the government with alarming details of our lives stripping us of the last remnants of privacy – this is the menace. For example, I can use Big Data predictive analytics to forecast the movement of the equity markets, recommend products you are likely to purchase, or recommend a new favorite movie.  On the other hand as the links below demonstrate Big Data provides the tools for privacy invasion at a scale never before seen:
 
Public Data and the Derivation of Social Security Numbers
Rethinking Privacy in an Era of Big Data
Big data collection collides with privacy concerns, analysts say
Microsoft’s Charney warns of big data privacy, security challenges
Privacy in the Age of Big Data
Big Data Brings Big Privacy Concerns
How “Big Data” Can Predict Your Divorce

Education: A Lifelong Path

It is; and, has been, my belief that we are individually responsible for maintaining our capabilities in an ever-changing employment landscape in order to avoid obsolescence (i.e. unemployment). Continuous training is much more relevant for technical professionals where rapid technological evolution is the norm lest we find ourselves pigeon-holed into a dying niche.  The tools for continuous training can be found in reading a well-written book, participating in a web-based course with a live instructor, attending an in-person instructor-led class, or watching a well-constructed computer-based training module (Supporting Life-Long Learning with Constructivist Web-based Instruction).  The key principle in any method of delivery is that the participant is learning, expanding their understanding, and maintaining their relevancy in an ever-changing world.  Moreover, training is an investment, an investment with an enviable return-on-investment (ROI) as evidenced by research findings: “Companies in the top quarter in training expenditure per employee per year ($1,500 or more) average 24% higher profit margins than companies that spend less per year.”(HR Magazine)

So, what does all this mean for a SQL Server professional?  Recently, I was asked to rate my SQL Server abilities there was an awkward silence when I said “7″, maybe “8″ on a good day.  When my counterpart described himself as a “10″ I had to explain my response by saying that I believe the SQL Server domain is so vast that I did not feel that I could judge myself an expert in all aspects of the system.  I think this may be true for many technology professionals (obviously, my counterpart is the exception) which provides even those of us who work in current technologies with opportunities to learn.

Data Replication Paper published on SQLServerCentral.com

The goal of this paper is to provide a foundation for understanding data replication as well as a discussion of the criteria for selecting an appropriate replication technology.  Read it on SQLServerCentral.

If you don’t have a SQLServerCentral account then read it here.

Data Insecurity

As a DBA and former software engineer one of my recurring nightmares involves the release of sensitive information from a system for which I am responsible.  Every day it seems I read of another incident of data breach; that is, “the intentional or unintentional release of secure information to an untrusted environment.”  The list below represents the data breach activity that has been reported for the past ten days (source:Privacy Rights Clearinghouse):

October 18, 2012	Blount Memorial Hospital Maryville, Tennessee
A password-protected laptop was stolen from an employee’s home on August 25.  It contained two groups of patient data.  Patient names, dates of birth, responsible party names, patient addresses, physician names, and billing information for 22,000 patients were on the laptop. An additional 5,000 patients had similar information exposed as well as their Social Security numbers and other non-medical information.
October 18, 2012	Southern Environmental Law Center Charlottesville, Virginia
Sensitive information from Southern Environmental Law Center was placed online. Credit card, medical, and donor information such as addresses, phone numbers, and client files were exposed. The data was accessible via Google search for an unspecified amount of time. Southern Environmental Law Center is warning people not to open emails about the security failure or click on any links in emails that appear to be from Southern Environmental Law Center.
October 15, 2012	District 202 Plainfield, Illinois
People who applied online for a job in District 202 had their information accessed by a hacker. The hacker sent messages to former and current job applicants and informed them that the Plainfield School District 202 website was breached.
October 13, 2012	City of Burlington, Washington Burlington, Washington
A hacker or hackers managed to transfer $400,000 in city funds to accounts across the country. The cyber attack occurred sometime between Tuesday night and Wednesday morning.  City employees may have also had their direct deposit bank account information compromised.
October 12, 2012	AutoCarry North Bergen, New Jersey
An office burglary that occurred on October 10 resulted in the exposure of customer information.  Paper documents that contained credit card numbers, addresses, and other personal information were taken.
October 12, 2012	Korn/Ferry International Los Angeles, California
A cyber breach affected Korn/Ferry databases.  Names, Social Security numbers, driver’s license numbers, government-issued identification numbers, credit card numbers, and health information may have been exposed.  The information may have been available to unauthorized parties for months before the breach was discovered in August of 2012.
October 11, 2012	Centers for Medicare & Medicaid Services (CMS) Baltimore, Maryland
The CMS experienced 13 breaches between September 23, 2009 and December 31, 2011.  The CMS failed to notify beneficiaries of seven of the breaches in a timely manner.  The HHS’s Office of the Inspector General (OIG) also alleges that the notifications mailed to beneficiaries did not disclose what type of information had been exposed, the date the breach occurred, or how CMS was working to prevent future breaches.
October 10, 2012	Northwest Florida State College Niceville, Florida
An internal review revealed a hack of Northwest College servers.  One or more hackers accessed at least one folder in the server between May 21, 2012 and September 24, 2012.  Over 3,000 employees, 76,000 Northwest College student records, and 200,000 students eligible for Bright Future scholarships in 2005-06 and 2006-07 were affected.  Bright Future scholarship data included names, Social Security numbers, dates of birth, ethnicity, and genders.  Current and former employees that have used direct deposit anytime since 2002 may have had some information exposed. At least 50 employees had enough information in the folder to be at risk for identity theft.
October 8, 2012	TD Bank Cherry Hill, New Jersey
Two data backup tapes were lost during shipping in late March 2012.  The tapes included customer names, Social Security numbers, addresses, account numbers, debit card numbers, and credit card numbers.

“Used with the permission of the Privacy Rights Clearinghouse, http://www.privacyrights.org.”;

The fear mongering about data breaches most commonly involves “Identity Theft” where a “Hacker” uses the obtained personal information to assume the identity of an unsuspecting individual whose Social Security Number and other information has been “stolen”. Clearly, data breaches make the process of obtaining personal information about any individual easier; however, the fact remains that anyone’s personal information is obtainable WITHOUT access to any restricted data via a variety of public Internet sources.

Notwithstanding the identity theft hyperbole, as a DBA, it is my responsibility to ensure that my employer NEVER makes the Data Breach list. To that end I am a disciple of, and evangelist for, encryption of all sensitive data stored in my databases. I discuss the various encryption options for SQL Server in the paper Data Insecurity: A Perspective on Data Encryption.

Contained Databases in SQL Server 2012

The process of database migration is now easier than ever with the Contained Database feature in SQL Server 2012.  A Contained Database (properly referred to as a Partially Contained Database) has become a fully encapsulated object providing definitive separation of the database and the instance.  The containment of database objects and configuration metadata allows for moving a database in a single step. The image below demonstrates the encapsulation of database specific metadata in the Contained Database:

Contained databases support both Windows and SQL Server authentication by allowing the creation of Windows authenticated logins using the same process as previous versions of SQL Server as well as Contained Logins created within the Contained Database.  The Contained Login is NOT stored in master.sys.logins.  The authentication process for Windows-based logins uses the same process as previous versions of SQL Server while Contained Logins are authenticated against the Contained database first.  If the Contained login is not found in the Contained database, then the authentication is performed at the instance level; however, if the Contained login fails at the Contained database level because of an incorrect password, then the connection is terminated.

As with any new feature there are benefits and risks associated with the use of Contained databases; this missive includes a brief introduction to the benefits leaving the more detailed discussion to another medium.

Watch my Contained Database presentation.

Woz weighs in on the Cloud.

Apparently my evangelism against the Cloud is taking root as evidenced by Steve Wozniak’s recent comments.

Woz says:

“I really worry about everything going to the cloud,” he said. “I think it’s going to be horrendous. I think there are going to be a lot of horrible problems in the next five years.”

He added: “With the cloud, you don’t own anything. You already signed it away” through the legalistic terms of service with a cloud provider that computer users must agree to.

“I want to feel that I own things,” Wozniak said. “A lot of people feel, ‘Oh, everything is really on my computer,’ but I say the more we transfer everything onto the web, onto the cloud, the less we’re going to have control over it.”

SQLSaturday #158

A large crowd and awesome presentations made this free event the place to be on Saturday.  A personal thank you to all who attended my presentation.  See you at the next event.

SQL Server Saturday #158 NYC on August 4th

I am proud to announce that I will be presenting at SQLSaturday on August 4th at Microsoft New York Metro Office, 1290 Avenue of the Americas, Sixth Floor, New York, NY 10104.

SQLSaturday is a training event for SQL Server professionals and those wanting to learn about SQL Server.  More information is available at the SQLSaturday #158 event page.

Posted a paper discussing SQL Server Replication

Read the paper here.

SQL Server Saturday #121 Presentation

A presentation discussing an easy approach for the creation of a performance baseline for a SQL Server instance that I presented at the SQL Saturday event in Philadelphia.

Cloud Computing Redux

Beyond the security, ownership, and other issues surrounding cloud computing comes this missive:

The Hidden Risk of a Meltdown in the Cloud

It has been my opinion that cloud computing has its benefits and weaknesses (see post below); however, those benefits do not extend to data storage or for systems without redundant non-cloud-based backup.

IO, IO it’s off to work we go

When we focus on IO performance for SQL Server it is vitally important to understand the architecture of the disk subsystem. Typically, a DBA will focus on Disk Queue Length as the primary measure of IO performance; however, in a Storage Area Network (SAN) environment the Disk Queue Length metric may not provide reliable results. In a SAN environment a “disk” as recognized by SQL Server, may not be composed of a single physical spindle (physical disk); therefore, the Disk Queue Length metric must be considered in relation to the actual number of spindles. For example, a “Logical Disk” labeled “M:\” may actually be composed of four (4) spindles meaning that a Disk Queue Length value of twelve (12) (which is, in my opinion, the threshold for identifying disk IO issues) must be normalized to the number of spindles because the metric represents the total Disk Queue Length:

Disk Queue Length / Spindles = Physical Disk Queue Length.

In our example resulting in a value of:

12 / 4 = 3.

A Disk Queue Length of three (3) does not represent a negative performance metric.

With the discussion above in mind we address SAN-based SQL Server IO performance using a different metric which may be more reliable – Disk Latency / Data Transfer Time. Using the Disk Latency metric permits the DBA to abstract the underlying disk subsystem allowing direct measurement of the performance of the disks as reported to the operating system. In my opinion, the Disk Latency for a well-configured disk subsystem should be less than four milliseconds (4 ms).

SSISy Scripting

I’ve done a fair bit of work implementing custom ETL solutions with SSIS concluding that the tool provides excellent “out-of-the-box” support for ETL as well as almost limitlessly extensibility with the Script component.  If you have a need to bring data from there to here with transformations along the way then SSIS is the solution; full transactional support, easy to use, fast, and extensible.

I use C# to develop my Script components.  As a simple example, during development I sometimes use a dialog to display running values within the package.

Notice that the plumbing has been generated by the system allowing us to focus on functionality.  All of the variables are addressable within the script with read / write access defined in the Script Component’s properties dialog in BIDS.  Notice the call to display the MessageBox (look familiar?):

public void Main()
{
MessageBox.Show(Dts.Variables["RowCount"].Value.ToString() + “/” +                        
Dts.Variables["UnmatchedRowCount"].Value.ToString(), “RowCount”, MessageBoxButtons.OK, MessageBoxIcon.Exclamation);

Dts.TaskResult = (int)ScriptResults.Success;
}

There isn’t much that can’t be done with a Script Component; on occasion I’ll simplify a Data Flow by encapsulating multiple data transforms or cleansing operations in a script.  Even if you are a magician with the built-in tasks and components in SSIS I think you’ll find that using the Script Component will simplify and extend the capabilities of your package.