EMC Avamar Deduplication

EMC Avamar enables fast, efficient backup and recovery by reducing the size of backup data at the client—before it’s transferred across the network and stored. Avamar's variable-length deduplication dramatically reduces network traffic by only sending unique blocks, compressed and encrypted over local area networks (LANs) or wide area networks (WANs). Blocks that were previously stored are never backed up again.

Avamar is a sourcebase global deduplication solution which enables fast, efficient backup and recovery by reducing the size of backup data at the client before it is transferred across the network and stored. Avamar’s data deduplication dramatically reduces network traffic by only sending unique blocks over LAN/WAN. Blocks that were previously stored are never backed up again.

This means huge savings in bandwidth for backup, a lot less disk storage needed on the back end, and most importantly very fast backups often as much as ten times faster.

Avamar backups can be quickly recovered in just one step eliminating the hassle of restoring full and subsequent incremental backups to reach the desired recovery point. Backup data is encrypted during transit across the network and at rest for added security.

There are three ways to define data deduplication:

File-Level 
File level deduplication reduce storage needs for file servers by identifying duplicate files and providing an efficient mechanism for consolidating them. The most common implementation of single instance storage is at the file level. With this method, a single change in a file results in the entire file being identified as unique. Example: If there were 3 versions of a file in a backup environment, the 3 files in their entirety are stored.

Fixed Block 
Fixed block deduplication is commonly employed in snapshot and replication technologies. This method breaks a file into fixed length sub-objects. However, even with small changes to the data, all fixed length segments in a dataset can change despite the fact that very little of the dataset has actually changed.

Variable Block 
With variable block level deduplication, a change in a file results in only the variable-sized block containing the change being identified as unique. Consequently, more data is identified as common data, and in the case of backup, there is less data to store as only the unique data is backed up. This is the method used by Avamar

NetApp Common Terminology

NetApp Common Terminology  - Storage Terms

storage controller
The component of a storage system that runs the Data ONTAP operating system and controls its disk subsystem. Storage controllers are also sometimes called controllers, storage appliances, appliances, storage engines, heads, CPU modules, or controller modules.

storage system
The hardware device running Data ONTAP that receives data from and sends data to native disk shelves, third-party storage, or both. Storage systems that run Data
ONTAP are sometimes referred to as filers, appliances, storage appliances, VSeries systems, or systems.

Cluster and high-availability terms below:

cluster

• In Data ONTAP 8.x Cluster-Mode, a group of connected nodes (storage systems) that share a global namespace and that you can manage as a single virtual server or multiple virtual servers, providing performance, reliability, and scalability benefits.
• In the Data ONTAP 7.1 release family and earlier releases, a pair of storage systems (sometimes called nodes) configured to serve data for each other if one of the two systems stops functioning.
• In the Data ONTAP 7.3 and 7.2 release families, this functionality is referred to as an active/active configuration.
• For some storage array vendors, cluster refers to the hardware component on which host adapters and ports are located. Some storage array vendors refer to this component as a controller.

HA (high availability)

• In Data ONTAP 8.x, the recovery capability provided by a pair of nodes (storage systems), called an HA pair, that are configured to serve data for each other if one of the two nodes stops functioning.
• In the Data ONTAP 7.3 and 7.2 release families, this functionality is referred to as an active/active configuration.

HA pair 

• In Data ONTAP 8.x, a pair of nodes (storage systems) configured to serve data for each other if one of the two nodes stops functioning.
• In the Data ONTAP 7.3 and 7.2 release families, this functionality is referred to as an active/active configuration.

What is NetApp Autosupport

What is NetApp AutoSupport?

NetApp AutoSupport is an integrated and efficient monitoring and reporting technology that checks the health of AutoSupport-enabled NetApp systems on a continual basis. It’s an effective troubleshooting tool for you and for the NetApp Support team. It provides capability to configure filers to initiate automatic sending of email notifications to NetApp technical support and other designated addressees when specific events occur

Why ?
Helps solve problems proactively

How ?

• Via email (smtp, preferred) or the web (http(s))
• Configure via the web interface or
• Via the 'options' command

filer> options autosupport.support.transport smtp
filer> options autosupport.doit testing123

If the email subject contains the word “test”, NetApp will send a reply to all email recipients.

What is Avamar - EMC's top selling backup solution

EMC's top selling backup solution Avamar has a refreshing new twist on the backup world. Avamar utilizes backup to disk hardware and client software installs to create a client-server network backup/restore solution while using a unique de-dupe technology.

Traditional backup solutions have some significantly inherent problems. A high-percentage of data that is retained on backup solutions is redundant data. Typical strategies consist of daily incremental backups and weekly full backups. These backups can be very time consuming, especially when getting the full backups, and yield multiple copies of identical or slowly changing data on backup media. This media then has to be organized and kept off-site for disaster recovery purposes, which often leads to high protection service costs, and high recovery time objectives. In addition, the data itself that is often duplicated across several servers in the case of system files, and many users keep identical or slightly different versions of the same documents. Traditional backups will backup all copies and variations as if they were brand new documents. Backing up redundant data increases the amount of backup storage needed and can negatively impact network bandwidth. The backup window for most organizations gets smaller and smaller the more they grow.

Avamar differs from traditional backup and restore solutions by identifying and storing only unique data objects. Redundant data is identified at the source, which reduces the amount of data that actually needs to travel to the backup server node. Avamar utilizes a chunk and hash methodology to break down all files into smaller chunks of data to send to the backup node. Unique hashes are generated from this data that gets stored on the backup node. If a client tries to send the same data, the backup node will respond that it already has that data and does not need another copy. Only new, modified, or changed chunks from the original files will be sent to the backup node.

Key Features:

• Global data de-duplication ensures that data objects are only backed up once across the backup environment.
• Systematic fault tolerance, using RAID, RAIN, checkpoints, and replication provides data integrity and disaster recovery protection.
• Highly reliable inexpensive disk storage for primary backup storage.
• Standard IP network technologies. Optimizes use of network for backup, does not require a separate backup network.
• Centralized management. The Avamar Administrator and Enterprise consoles give users full-featured remote management of Avamar servers with robust reporting capabilities.

Support for Windows, Unix, Linux, NDMP, SQL, Exchange, DB2, and Oracle.
So now that you have some of the basics, check back soon to get a more in depth look to how the different solutions are deployed. There are 3 basic versions:

Avamar Data Store Non-RAIN (single or multiple nodes with replication)
Avamar Data Store RAIN (multiple data nodes for fault tolerance)
Avamar Virtual Edition (virtual appliances versions in .5 and 1 TB)

rdfile command

rdfile (read), wrfile (write)

rdfile is a NetApp command is to read a file in NetApp environment.  Example below to read the quota file.

What is Data ONTAP Cluster-Mode?

Data ONTAP 8.1 Cluster-Mode is based on the NetApp high-availability (HA) pair technology. Data ONTAP 8.1 Cluster-Mode includes a transformation of the layers of virtualization, the addition of a cluster-network path, virtualized storage containers that can span storage boundaries, and greater virtualized networking access.

All of these features map to the three key benefits:
Always-on, 
on-demand flexibility, and
Operational Flexibility

1st day with NetApp

Started on my first journey with NetApp learning today.  Signed up at http://now.netapp.com.  It's easy and takes a few minutes to do so.

Next, I went to sign up for the course called:

Data ONTAP 7-Mode Fundamentals

What is Data ONTAP 7-Mode?

First, lets go back to NetApp's operating system simply called Data ONTAP (latest version is 8 onwards) and under it, there are 2 types just like we have Windows 7 and Windows 8:'
- 7-mode
- Cluster-mode

Brief overview:
Data ONTAP 8 is the successor to Data ONTAP 7G and the culmination of years of storage technology development at NetApp. It was introduced in September 2009 and reached general availability (GA) status in March 2010. The latest version, Data ONTAP 8.0.1, was released in September 2010. (Note that when using the phrase Data ONTAP 8, I’m referring to all existing and future releases in the line.)
Data ONTAP 8 merges the capabilities of two previously separate branches of Data ONTAP—Data ONTAP 7G and Data ONTAP GX—into a single code base that has two distinct operating modes:

7-Mode. As the name implies, 7-Mode delivers capabilities equivalent to the Data ONTAP 7.3.x releases with the same user interface and management tool support. Systems running Data ONTAP 7.3.x releases can upgrade to 7-Mode nondisruptively.

Cluster-Mode. This mode inherits and improves upon the capabilities of Data ONTAP GX, its predecessor, clustered OS for scale-out storage. Cluster-Mode supports multicontroller configurations with a global namespace and clustered file system.