Kerberos这一名词来源于希腊神话 三个头的狗——地狱之门守护者
Kerberos /ˈkərbərəs/ is a computer network authentication protocol which works on the basis of 'tickets' to allow nodes communicating over a non-secure network to prove their identity to one another in a secure manner.
Authentication
is the act of confirming the truth of an attribute of a single piece of data claimed true by an entity.
Authorization
is the function of specifying access rights to resources related to information security and computer security in general and to access control in particular.
Authentication is about who somebody is.
Authorisation is about what they're allowed to do.
TGT: Ticket Granting Ticket From: The MIT Kerberos Administrator’s How-to Guide
TGS: Ticket Granting Service
From: Hadoop Security Design
Proglem: Applications on Yarn cannot run > 7 days
Problem: cannot run > 7 days on Yarn
From: Long Running Spark Apps on Secure YARN
Upload Keytab to HDFS and login from Keytab before token expired
Spark-5342 Allow long running Spark apps to run on secure YARN/HDFS
Configuring YARN for Long-running Applications (Hadoop 2.6.0)
1 // Log a user in from a keytab file. Loads a user identity from a keytab
2 // file and logs them in. They become the currently logged-in user.
3 static void loginUserFromKeytab(String user, String path)
4
5 // Log a user in from a keytab file. Loads a user identity from a keytab
6 // file and login them in. This new user does not affect the currently
7 static UserGroupInformation loginUserFromKeytabAndReturnUGI(String user,
8 String path)
9
10 // Return the current user, including any doAs in the current stack.
11 static UserGroupInformation getCurrentUser()
12
13 // Re-login a user from keytab if TGT is expired or is close to expiry.
14 void checkTGTAndReloginFromKeytab()
15
16 // Add the given Credentials to this user.
17 public void addCredentials(Credentials credentials)
18
19 public Credentials getCredentials()
20
21 // Run the given action as the user.
22 public <T> T doAs(PrivilegedAction<T> action)
1 UserGroupInformation.loginUserFromKeytab(principal, keytab)
2 val ugi = UserGroupInformation.getCurrentUser
A Subject represents a grouping of related information for a single entity, such as a person. Such information includes the Subject's identities as well as its security-related attributes (passwords and cryptographic keys, for example).
1 val ugi = UserGroupInformation.getCurrentUser
1 /**
2 * Perform work as a particular Subject.
3 *
4 * This method first retrieves the current Thread's
5 * AccessControlContext via AccessController.getContext,
6 * and then instantiates a new AccessControlContext
7 * using the retrieved context along with a new
8 * SubjectDomainCombiner (constructed using the provided Subject).
9 * Finally, this method invokes AccessController.doPrivileged,
10 * passing it the provided PrivilegedAction,
11 * as well as the newly constructed AccessControlContext.
12 *
13 * @param subject the Subject that the specified
14 * action will run as. This parameter may be null.
15 */
16 public static <T> T doAs(final Subject subject,
17 final java.security.PrivilegedAction<T> action)
1 System.setProperty("java.security.krb5.realm", "HADOOP.QIYI.COM")
2 System.setProperty("java.security.krb5.kdc", "hadoop-kdc01")
Run once every day in crontab
1 kinit -l 3d -k -t ~/test.keytab test@HADOOP.QIYI.COM
Right:
1 val fs = FileSystem.get(new Configuration())
2 while (true) {
3 println(fs.listFiles(new Path("/user"), false))
4 Thread.sleep(60 * 1000)
5 }
Kerberos will relogin automatically, when token is expired
Right:
1 UserGroupInformation.loginUserFromKeytab(principal, keytab)
2 val fs = FileSystem.get(new Configuration())
3 while (true) {
4 println(fs.listFiles(new Path("/user"), false))
5 Thread.sleep(60 * 1000)
6 }
Kerberos will relogin automatically, when token is expired
Wrong:
1 UserGroupInformation.loginUserFromKeytab(principal, keytab)
2 val fs = FileSystem.get(new Configuration())
3 while (true) {
4 UserGroupInformation.loginUserFromKeytab(principal, keytab)
5 println(fs.listFiles(new Path("/user"), false))
6 Thread.sleep(60 * 1000)
7 }
Do NOT call UserGroupInformation.loginUserFromKeytab again
Wrong:
1 val ugi = UserGroupInformation.loginUserFromKeytabAndReturnUGI(principal,
2 keytab)
3 ugi.doAs(new PrivilegedExceptionAction[Void] {
4 override def run(): Void = {
5 val fs = FileSystem.get(new Configuration())
6 while (true) {
7 println(fs.listFiles(new Path("/user"), false))
8 Thread.sleep(60 * 1000)
9 }
10 null
11 }
12 })
Kerberos will NOT relogin automatically, when token is expired
Right:
1 val ugi = UserGroupInformation.loginUserFromKeytabAndReturnUGI(principal,
2 keytab)
3 ugi.doAs(new PrivilegedExceptionAction[Void] {
4 override def run(): Void = {
5 val fs = FileSystem.get(new Configuration())
6 while (true) {
7 UserGroupInformation.getCurrentUser.reloginFromKeytab()
8 println(fs.listFiles(new Path("/user"), false))
9 Thread.sleep(60 * 1000)
10 }
11 null
12 }
13 })
Call reloginFromKeytab before token is expired
Right:
1 val creds = new org.apache.hadoop.security.Credentials()
2 val ugi = UserGroupInformation.loginUserFromKeytabAndReturnUGI(principal,
3 keytab)
4 ugi.doAs(new PrivilegedExceptionAction[Void] {
5 override def run(): Void = {
6 val fs = FileSystem.get(new Configuration())
7 fs.addDelegationTokens("test", creds)
8 null
9 }
10 })
11 UserGroupInformation.getCurrentUser.addCredentials(creds)
Update credentials periodically before token expired
Table of Contents | t |
---|---|
Exposé | ESC |
Full screen slides | e |
Presenter View | p |
Source Files | s |
Slide Numbers | n |
Toggle screen blanking | b |
Show/hide slide context | c |
Notes | 2 |
Help | h |