DCache Storage Endpoints

From ESCAPE_WIKI
Jump to: navigation, search

ESCAPE dCache

This page provides information for ESCAPE "users", particularly domain experts and people working on integration. The intention is to provide complete working examples on how you can interact with storage directly. This may be directly useful, but may also allow for diagnostic work.

Getting started

OIDC-Agent

https://github.com/indigo-dc/oidc-agent

Rclone

Rclone is a command line program to sync files and directories to and from.

Configuration
Rclone needs some variables set in order to work. name, type of storage, vendor, user, pass, bearer_token_command etc.

Storage providers
WebDAV, Google Drive, Amazon S3, Microsoft OneDrive etc.

Commands
ncdu, sync, copy, move, link, ...

https://rclone.org/

WebDAV

WebDAV is an extension of the Hypertext Transfer Protocol (HTTP) that allows systems to read and write documents on the web.

https://rclone.org/webdav/

DESY Endpoints

Prometheus and dCache Demo with GUI support are available for everyone/sites participating in the Datalake testbed and using dCache as the storage topology.

Properties Prometheus dCache-demo
Maintenance mode Data is wiped and rebuild every day Well-maintained instance with a reasonable capacity
New features As soon as available Possible short delay
Stability Unstable and flaky Stable and Reliable
WebDAV URL https://prometheus.desy.de https://dcache-demo.desy.de:2443
GUI link https://prometheus.desy.de:3880 https://dcache-demo.desy.de
Grid-WebDav dcache-se-doma.desy.de:2880
Grid-FTP dcache-se-doma.desy.de:2811
Gsi-DCap dcache-se-doma.desy.de:22128
SRM dcache-se-doma.desy.de:8443
XrootD dcache-se-doma.desy.de:1094
Auth via X.509, OIDC, Sci tokens

IN2P3 Endpoints

Properties Nessie
Maintenance mode Well-maintained instance with a expandable capacity
New features As soon as needed
Stability Stable and Reliable
WebDAV URL https://ccdcalitest10.in2p3.fr:2880
GUI link https://ccnessie.in2p3.fr:3880/
XrootD root://ccdcalitest10.in2p3.fr:1094/pnfs/in2p3.fr/data/escape
GSI gsiftp://ccdcalitest10.in2p3.fr:2811/pnfs/in2p3.fr/data/escape
Auth via X.509, OIDC, Sci tokens

SURFsara Endpoints

Properties Dolphin
Maintenance mode Test, small capacity
New features As soon as needed
Stability Unstable
WebDAV URL https://dolphin12.grid.surfsara.nl:2880/
GUI link https://dolphin12.grid.surfsara.nl:20443/
XrootD dolphin12.grid.surfsara.nl:21094
GridFTP dolphin12.grid.surfsara.nl:2811
Auth X.509, OIDC, macaroons. At request: SciTokens, basic (username/passwd)