Opendedup

Global Deduplication

  • Increase font size
  • Default font size
  • Decrease font size

Volume Creation Man Page

E-mail Print PDF

usage: mkfs.sdfs --volume-name=sdfs --volume-capacity=100GB

 

--aws-enabled <true|false>                        Set to true to enable this volume to store to Amazon S3 Cloud Storage. cloud-secret-key, cloud-access-key, and  cloud-bucket-name will also need to be set.   

--azure-enabled <true|false>                      Set to true to enable this volume to store to Microsoft Azure Cloud Storage. cloud-secret-key, cloud-access-key, and cloud-bucket-name will also need to be set.

--base-path <PATH>                                The folder path for all volume data and meta data.  Defaults to: /opt/sdfs/<volume name>   

--chunk-store-compress <true|false>               Compress chunks before they are stored. By default this is set to true. Set it to  false for volumes that hold data that does not compress well, such as pictures and  movies   

--chunk-store-data-location <PATH>                The directory where chunks will be stored. Defaults to:  --base-path + /chunkstore/chunks   

--chunk-store-encrypt <true|false>                Whether or not to Encrypt chunks within the Dedup Storage Engine. The encryption key is generated automatically. For AWS and Azure  this is a good option to enable. The default for this is false   

--chunk-store-eviction <HOURS>                    The duration, in hours, that chunks will be removed from Dedup Storage Engine if unclaimed. This should happen less  frequently than the io-claim-chunks-schedule.  Defaults to: 6   

--chunk-store-gc-schedule <CRON Schedule>         The schedule, in cron format, to check for unclaimed chunks within the Dedup Storage Engine. This should happen less  frequently than the io-claim-chunks-schedule.  Defaults to:  0 0 0/2 * * ?   

--chunk-store-hashdb-class <class name>           The class used to store hash values Defaults to: org.opendedup.collections.FileBasedCSMap

--chunk-store-hashdb-location <PATH>              The directory where hash database for chunk locations will be stored. Defaults to: --base-path + /chunkstore/hdb   

--chunk-store-local <true|flase>                  enables or disables local chunk store. The chunk store can be local(true or remote(false) provided you supply the routing config file and there is a storageHub listening on the remote server(s) when you mount the SDFS volume.  Defaults to: true                                

--chunk-store-pre-allocate <true|false>           Pre-allocate the chunk store if true. Defaults to: false   

--chunk-store-size <MB|GB|TB>                     The size in MB,TB,GB of the Dedup Storeage Engine. This Defaults to the size of the Volume   

--chunkstore-class <Class Name>                   The class for the specific chunk store to be used.  Defaults to org.opendedup.sdfs.filestore.FileChunkStore

--cloud-access-key <Cloud Access Key>             Set to the value of Cloud Storage access key.

--cloud-bucket-name <Unique Cloud Bucket Name>    Set to the value of Cloud Storage bucket name. This will need to be unique and a could be set the the access key if all  else fails. aws-enabled, aws-secret-key, and aws-secret-key will also need to be set.   

--cloud-secret-key <Cloud Secret Key>             Set to the value of Cloud Storage secret key.   

--dedup-db-store <PATH>                           The folder path to location for the dedup file database. Defaults to:  --base-path + /ddb   

--dse-enable-network                              Enable Network Services for Dedup Storage Enginge to serve outside hosts   

--dse-enable-udp                                  Enable udp for some communication between Volume and DSE. Defaults to false   

--dse-listen-ip <IPv4 Address>                    Host name or IPv4 Address to listen on for incoming connections. Defaults to "0.0.0.0"   

--dse-listen-port <IP Port>                       TCP and UDP Port to listen on for incoming connections. Defaults to 2222   

--dse-upstream-enabled                            Enable Upstream Dedup Storage Engine communication   

--dse-upstream-host <FQDN or IPv4 Address>        Host name or IPv4 Address of upstream dse   

--dse-upstream-host-port <IP Port>                TCP and UDP Port to listen on for incoming connections. Defaults to 2222   

--dse-upstream-password <STRING>                  SDFSCLI Password of upstream host. Defaults to "admin"   

--enable-replication-master                       Enable this volume as a replication master   

--enable-replication-slave                        Enable this volume as a replication slave   

--gc-class <CLASS NAME>                           The class used for intelligent block garbage collection. Defaults to: org.opendedup.sdfs.filestore.gc.PFullGC

--gs-enabled <true|false>                         Set to true to enable this volume to store to Google Cloud Storage. cloud-secret-key, cloud-access-key, and cloud-bucket-name will also need to be set.   

--hash-type <tiger16|tiger24|murmur3_128>         This is the type of hash engine used to calculate a unique hash. The valid options for hash-type are tiger16 tiger24  murmur3_128 This Defaults to murmur3_128   

--help                                            Display these options.   

--io-chunk-size <SIZE in kB>                      The unit size, in kB, of chunks stored. Set this to 4 if you would like to dedup VMDK files inline.  Defaults to 128

--io-claim-chunks-schedule <CRON Schedule>        The schedule, in cron format, to claim deduped chunks with the Dedup Storage Engine. This should happen more frequently than the chunk-store-gc-schedule. Defaults to:  0 0 0/1 * * ?  

--io-file-read-cache <NUMBER>                     The number of memory buffers to have available for reading per file. Each buffer in the size of io-chunk-size.  Defaults to 1   

--io-log <PATH>                                   The file path to location for the io log. Defaults to   --base-path + /sdfs.log   

--io-max-file-write-buffers <SIZE in MB>          The amount of memory to have available for reading and writing per file. Each buffer in the size of io-chunk-size. Defaults to 1   

--io-max-open-files <NUMBER>                      The maximum number of files that can be open at any one time. If the number of files is exceeded the least recently used will be closed. Defaults to: 1024   

--io-meta-file-cache <NUMBER>                     The maximum number metadata files to be cached at any one time. If the number of files is exceeded the least recently used will be closed. Defaults to: 1024  

--io-multi-read-timeout <NUMBER>                  Timeout to try to read from cache before it request data from the chunkstore. Defaults to: 1000   

--io-safe-close <true|false>                      If true all files will be closed on filesystem close call. Otherwise, files will be closed based on inactivity. Set this  to false if you plan on sharing the file system over an nfs share. Defaults to true   

--io-safe-sync <true|false>                       If true all files will sync locally on filesystem sync call. Otherwise, by defaule (false), files will sync on close and data will per written to disk based on

--max-file-write-buffers.  Setting this to true will ensure that no data loss will occur if the system is turned off abrubtly at the cost of slower speed.  Defaults to: false  

--io-system-read-cache <NUMBER>                   Size, in number of chunks, that read chunks will be cached into memory. Defaults to:  1000   

--io-write-threads <NUMBER>                       The number of threads that can be used to process data writted to the file system.  Defaults to: 16   

--permissions-file <POSIX PERMISSIONS>            Default File Permissions. Defaults to:     0644   

--permissions-folder <POSIX PERMISSIONS>          Default Folder Permissions.   Defaults to: 0755   

--permissions-group <POSIX PERMISSIONS>           Default Group.   Defaults to: 0   

--permissions-owner <POSIX PERMISSIONS>           Default Owner.  Defaults to:   0 (root)  

--replication-master <FQDN or IPv4 Address>       The Replication master for this slave  

--replication-master-password <STRING>            The Replication master sdfscli password. Defaults to "admin"   

--report-dse-capacity <true|false>                If set to "true" this volume will report capacity the actualcapacity statistics from the DSE. If this value is set to "false" it will report as virtual size of the volume and files. Defaults to "true"   

--report-dse-size <true|false>                    If set to "true" this volume will used as the actual used statistics from the DSE. If this value is set to "false" it  will report as virtual size of the volume and files. Defaults to "true"   

--sdfscli-listen-addr <ip address or host name>   IP Listenting address for the sdfscli management interface. This defaults to "localhost"   

--sdfscli-listen-port <tcp port>                  TCP/IP Listenting port for the sdfscli management interface   

--sdfscli-password <password>                     The password used to authenticate to the sdfscli management interface. Thee default password is "admin".   

--sdfscli-require-auth                            Require authentication to connect to the sdfscli managment interface   

--use-perf-mon <true|false>                       If set to "true" this volume will log io statistics to /etc/sdfs/ directory. Defaults to "false"   

--volume-capacity <SIZE [MB|GB|TB]>               Capacity of the volume in [MB|GB|TB].  THIS IS A REQUIRED OPTION   

-volume-maximum-full-percentage <PERCENTAGE>     The maximum percentage of the volume capacity, as set by volume-capacity, before the volume startsreporting that the disk  is full. If the number is negative then it will be infinite. This defaults to 95  e.g. --volume-maximum-full-percentage=95   

--volume-name <STRING>                            The name of the volume. THIS IS A REQUIRED OPTION

Last Updated on Saturday, 23 March 2013 23:20  

SDFS Info

Latest News

SDFS Version 2.0.1 has been release with minor fixes.
 
SDFS Version 2.0 Released after over a year of development. Download it.