Showing posts with label Storage. Show all posts
Showing posts with label Storage. Show all posts

RAID vs Erasure Coding

http://www.networkcomputing.com/storage/raid-vs-erasure-coding/1792588127
Posted by
Labels:

File System Comparison

https://zh.wikipedia.org/wiki/%E6%96%87%E4%BB%B6%E7%B3%BB%E7%BB%9F%E7%9A%84%E5%AF%B9%E6%AF%94
Posted by
Labels:

RAID Penalty

Each disk provide a certain amount of IO based off of the
  • Rotational Speed
  • Average Latency
  • Average Seek Time(*)

Read Operation: NO Penalty

Write Operation:
RAID    Write Penalty
0            1(NO Penalty)
1            2
5            4(2R, 2W)
6            6(3R, 3W)
10          2


Reference: http://theithollow.com/2012/03/21/understanding-raid-penalty/



Posted by
Labels:

Storage Overview

Computer Data Storage

http://en.wikipedia.org/wiki/Computer_data_storage


Parity bit

http://en.wikipedia.org/wiki/Parity_bit

Single-ended signaling v.s. Differential signaling




SAMBA (SMB - Sever Message Block)

一個讓 Windows 系統存取 Linux 檔案系統的 protocol, 鳥哥對此有很好的介紹. 他也提到, CIFS (Common Internet File System) 和 NFS (Network File System) 只在 Windows 或 Unix 裡面互通。不過，NFS 僅能讓 Unix 機器溝通， CIFS 只能讓 Windows 機器溝通。

Scale out & Scale up

Scale Out (Scale horizontally)橫向擴展，向外擴展
e.g. 我們向原有的web, 郵件系統添加一個新機器
Scale Up (Scale vertically)縱向擴展，向上擴展
e.g. 我們向原有的機器添加CPU, 內存

無論是Scale out, Scale up, Scale in，實際上就是一種架構的概念，這些概念可以用在儲存上, 數據庫上, 網路上等。

Storage Virtualization



Storage Performance Council (SPC-1)



Storage Service


Storage Architecture

  • DAS
    • A external storage system directly attached to a server without a storage network in between
    • Interface : SCSI, SAS, , iSCSI, FC, PCI-E
    • Block-Level access
    • Internal DAS
    • External DAS
  • SAN
    • A network of storage devices that can be accessed by multiple servers
    • Block-Level access
    • FC SAN
    • iSCSI SAN
    • Hybrid SAN
    • Others
  • NAS


FC-SAN v.s. IP SAN
  • Performance
    • High v.s. Low
  • TCO (Total cost of Ownership)
    • High v.s. Low
  • Manager Expertise
    • High v.s. Low
  • Host Components
    • FC HBA v.s. iSCSI HBA or NIC + software initiators
  • Networking Components
    • FC Switch, FC Cables v.s. Ethernet Switch, Ethernet Cables
  • Storage Components
    • FC-Host Storage v.s. iSCSI Host Storage

RAID Levels

  • See 儲存產品暨產業基本概要
  • JBOD (Just a Bunch Of Disks)
  • NRAID (Non-RAID, SPAN)
    • Link all of the drives in a sequence
  • RAID 0
    • Striping
  • RAID 1
    • Mirroring
  • RAID 3
    • Disk Striping with Dedicated Parity Disk
  • RAID 5
    • Disk Striping with Interspersed Parity Disk
  • RAID 6
    • Disk Striping with Two Interspersed Parity Disk
  • RAID 10
  • RAID 30
  • RAID 50

Physical Drive, Logical Drive, Logical Volume (Pool)

e.g. A physical drive partitioned into three logical drive C:, D:, E:

Logical Drive (LD): Simply an array of independent physical drives

Logical Volume (Pool)(LV) :  A LV is composed of one or several logical drives, the member logical drives can be the same RAID level or different RAID levels.

Partitions (Virtual Volume)(VV): The LV can be divided into partitions.

During operation, the hose sees
1) a non-partitioned LV or
2) a partition of a partitioned LV as one single physical drive


Cylinder, Head, Sector

http://en.wikipedia.org/wiki/Cylinder-head-sector

Write Back Cache v.s. Write Through Cache (Write v.s. Read)

WriteThrough:每次遇到寫入指令時都會真正將資料寫入記憶體中
WriteBack:遇到寫入指令時先不將資料寫入記憶體中,而是將 Cache 內儲存的該筆資料標示為已更改,等系統閒置時或該記憶體位址將被其他重複率更高的位址取代時,才真正將資料回寫
WriteThrough 的特點是效率較差,但安全性高,WriteBack 則正好相反反


Master Boot Record (MBR) v.s. GUID Partition Table (GPT)

http://www.petri.co.il/gpt-vs-mbr-based-disks.htm#

Multipath I/O

http://en.wikipedia.org/wiki/Multipath_I/O


Tagged Command Queuing (TCQ)

http://en.wikipedia.org/wiki/Tagged_Command_Queuing


Self-Monitoring, Analysis and Reporting Technology (S.M.A.R.T.)

http://en.wikipedia.org/wiki/S.M.A.R.T.


Fibre Channel

** Ports
=> A port in fibre channel terminology is any entity that actively communicates over the network, not necessarily a hardware port
=> http://en.wikipedia.org/wiki/Fibre_Channel#Ports

Topologies
=> FC_P2P : point to point
=> FC_AL : arbitrated loop
=> FC_SW : swiched fabric

AL_PA: Arbitrated Loop. Physical Address
=> Each device communicating on an arbitrated loop


World Wide Name (WWN)

=>   Each device node has a WWNN
=>   All ports share the WWNN, and own a unique WWPN
=>   World Wide Node Name (WWNN)
=>   World Wide Port Name (WWPN)
http://en.wikipedia.org/wiki/World_Wide_Name
http://ithelp.ithome.com.tw/question/10009960

Internet Storage Name Service (iSNS)


http://en.wikipedia.org/wiki/Internet_Storage_Name_Service


Clock synchronization

http://en.wikipedia.org/wiki/Clock_synchronization


SCSI command

http://en.wikipedia.org/wiki/SCSI_command


相關文件

Posted by
Labels:

Bus, SAN, DAS, and NAS

SCSI (Small Computer System Interface)(並列SCSI)



SCSI initiator and target

Initiator : Host (computer), client
Target : Data storage device (HDD), server
Initiator and Target : mainly for communicating with each other
Target includes LUN (Logic Units) : mainly for execution command
Reference: http://en.wikipedia.org/wiki/SCSI_initiator_and_target

trminator : 
Reference: http://en.wikipedia.org/wiki/SCSI_terminator

High voltage v.s. Low voltage
The difference between low voltage and high voltage for the same size cable is that cable size affects current carrying capacity, while power is voltage times current. As a result, the same size cable can transfer more power at high voltage than at low voltage. (Power = Voltage * Current, P = V * I)
This is why utility power transmission and distribution systems use high voltage. A typical transmission voltage in the US might be 69 kv, 138 kv, even 345 kv, and a typical distribution voltage in the US might by 13.2 kv.

Synchronous & Asynchronous Data transfer

SAS (Serial Attached SCSI)(串列SCSI)

由並列SCSI物理存儲介面演化而來。與並列方式相比，串列方式提供更快速的通訊傳輸速度以及更簡易的配置。此外SAS支援與串列ATA設備相容，且兩者可以使用相類似的電纜。SATA的硬碟可接在SAS的控制器使用，但SAS硬碟並不能接在SATA的控制器使用。

SAS是點對點（point-to-point）連接，並允許多個埠集中於單個控制器上，可以建立在主機板也可以另外添加。該技術建立在強大的並列SCSI通訊技術基礎上。SAS是採用SATA相容的電纜線採取點對點連接方式，從而在計算機系統中不需要建立菊花鏈（daisy-chaining）方式便可簡單地實現線纜安裝。
  • 第一代SAS為陣列中的每個驅動器提供3.0 Gbps（約300 MBps）的傳輸速率。
  • 第二代SAS為陣列中的每個驅動器提供6.0 Gbps（約600 MBps）的傳輸速率。

SAS由3種型別協定組成，根據連線的不同裝置使用相應的協定進行資料傳輸。
  • 序列SCSI協定 (SSP) — 用於傳輸SCSI命令。
  • SATA通道協定 (STP) — 用於傳輸SATA資料。
  • SCSI管理協定 (SMP) — 用於對SAS裝置的維護和管理。

iSCSI (發音為 /аɪskʌzi/)(IP-SAN = IP-Storage Area Network)

與傳統的SCSI技術比較起來，iSCSI技術有以下三個革命性的變化：
  • 把原來只用於本機的SCSI協同透過TCP/IP網路傳送，使連接距離可作無限的地域延伸。
  • 連接的伺服器數量無限（原來的SCSI-3的上限是15）。
  • 由於是伺服器架構，因此也可以實現在線擴容以至動態部署。

ATA (Advanced Technology Attachment)

(PATA = Parallel ATA)或稱(IDE = Integrated Drive Electronics)

SATA (Serial ATA)

是串列SCSI (SAS) 的孿生兄弟，兩者的排線相容，SATA硬碟可接上SAS介面。它是一種電腦匯流排，主要功能是用作主機板和大量儲存裝置（如硬碟及光碟機）之間的資料傳輸之用。SATA是已經完全取代舊式PATA（Parallel ATA或稱IDE）的新型硬碟介面，因採用序列方式傳輸資料而得名。現時，SATA分別有SATA 1.5Gbit/s、SATA 3Gbit/s和SATA 6Gbit/s三種規格。

SATA ports.jpg
主機板上的SATA 1.5Gb/s插口

頻寬速度換算

需要注意的是，在一些新的技術標準中，為了防止資料在高速傳輸中出錯而加入了校驗碼，比如PCI-E 2.0、USB 3.0和SATA 3.0中採用的是8/10編碼，每10位編碼中只有8位元是真實資料，這時單位換算就不再是1:8而是1:10了，USB 3.0的5Gbps速度實際上是理論500MB/s而非理論625MB/s，SATA 6Gbps的理論速度則是600MB/s而非750MB/s。 所謂3Gb/s的演算法，3000MHz的頻率 x 每次發送一個資料 x 80%(8b/10b的編碼) / 8 bits per byte = 300Mbytes/s，同理1.5Gb/s也是這樣可算成150MB/s，也就是一般我們在買硬碟時，有時候會看到SATA 150MB/s / 300MB/s，有時候又會看到SATA 1.5Gb/s / 3Gb/s的緣故。
以USB 3.0而言，它擁有5Gbps的頻寬，每次發送一個資料 x 80%(8b/10b的編碼) / 8 bits per byte = 500Mbytes/s，所以USB 3.0的頻寬比SATA 3.0的600MB/s 還來的小。
SATA版本頻寬速度
SATA 3.06Gb/s600MB/s
SATA 2.03Gb/s300MB/s
SATA 1.01.5Gb/s150MB/s
USB版本頻寬速度
USB 3.05Gb/s500MB/s




Fibre Channel



DAS

Refers to a digital storage system directly attached to a server or workstation, without a storage network in between.It is a retronym, mainly used to differentiate non-networked storage from SAN and NAS.

A typical DAS system is made of a data storage device (for example enclosures holding a number of hard disk drives) connected directly to a computer through a host bus adapter (HBA). Between those two points there is no network device (like hub, switch, or router), and this is the main characteristic of DAS.
The main protocols used for DAS connections are ATA, SATA, eSATA,[1] SCSI, SAS, and Fibre Channel.

SAN

出於歷史原因，數據中心中最初都是SCSI磁碟陣列的「孤島」群。每個單獨的小「島嶼」都是一個專門的直接連接存儲器應用，並且被視作無數個「虛擬硬碟驅動器」（例如LUNs）。本質上來說，SAN就是將一個個存儲「孤島」使用高速網路連接起來，這樣使得所有的應用可以訪問所有的磁碟。
作業系統會將SAN視為一組LUN，並且在LUN上維護自己的File System。這些不能在多個作業系統/主機之間進行共享的本地文件系統，具有非常高的可靠性和十分廣泛的應用。如果兩個獨立的本地文件系統存在於一個共享的LUN上，它們彼此沒有任何機制來知道對方的存在，沒有類似緩存同步(Distributed Lock Manager)的機制，所以可能發生數據丟失的情況。因此，在主機之間通過SAN共享數據，需要一些複雜的高級解決方案，例如SAN文件系統或者計算機集群。撇開這些問題，SAN對於提高存儲能力的應用有很大幫助，因為多個伺服器可以共享磁碟陣列上的存儲空間。SAN的一項典型應用是需要高速塊級別訪問的數據操作伺服器，比如電子郵件伺服器、資料庫、高利用率的文件伺服器等。

NAS

相對地，NAS允許多台計算機經過網路訪問同一個File system(Storage)，並且會自動同步它們的操作。由於NAS的引入使得SAN存儲可以被容易地轉換為NAS。
DAS、NAS 和 SAN 的比較

Daisy-chaining (菊花鏈)

Except for star-based networks, the easiest way to add more computers into a network is by daisy-chaining, or connecting each computer in series to the next. If a message is intended for a computer partway down the line, each system bounces it along in sequence until it reaches the destination. A daisy-chained network can take two basic forms: linear and ring.

 In electrical and electronic engineering a daisy chain is a wiring scheme in which multiple devices are wired together in sequence or in a ring. Other than a full, single loop, systems which contain internal loops cannot be called daisy chains.

  • Linear Topology
  • Ring Topology

相關文件

名稱頻寬 (Mbit/s)速度 (MB/s)接線最大長度 (m)電源供應每頻道最多可接設備
eSATA3,0003002 with eSATA HBA (1 with passive adapter)否 [2]1（15 with port multiplier）
串列SCSI3,0003758否4
SATA 6.0Gb/s6,0006001
SATA 3.0Gb/s3,0003001否1（15 with port multiplier）
SATA 1.5Gb/s1,5001501否1 per line
PATA 1331,0641330.46（18 英寸）否2
FireWire 32003,144393100; alternate cables available for 100 m+15 W, 12–25 V63 (with hub)
FireWire 80078698.25100[3]15 W, 12–25 V63 (with hub)
FireWire 40039349.134.5[3][4]15 W, 12–25 V63 (with hub)
USB 3.05,0005004.5 W, 5 V
USB 2.0480605[5]2.5 W, 5 V127 (with hub)
USB 1.1121.5
USB 1.01.50.1875
Ultra-640 SCSI5,12064012否15 (plus the HBA)
Ultra-320 SCSI2,560320
Fiber Channel
透過銅線		4,00040012否126
（16777216 with switches）
Fiber Channel
透過光纖		10,52020002–50000否126
（16777216 with switches）
Infiniband
12X Quad-rate		120,00012,0005 （銅線）[6][7]
<10000 nbsp="" p="">
否1 with Point to point
Many with switched fabric


Hub Switch Router Repeater Bridge
Posted by
Labels:
Subscribe to: Posts (Atom)