Bug #108919 acl_cache_lock is not consistent in acl_authenticate function for proxy_user
Submitted: 28 Oct 2022 14:19 Modified: 2 Nov 2022 16:01
Reporter: jingliang shang Email Updates:
Status: Can't repeat Impact on me:
None 
Category:MySQL Server: Security: Privileges Severity:S2 (Serious)
Version:8.0 OS:Any
Assigned to: CPU Architecture:Any
Tags: acl_authenticate, acl_cache_lock, proxy_user

[28 Oct 2022 14:19] jingliang shang
Description:
I'think that I've hit a race condition wrt the acl_cache_lock inconsistency.

There is possible race between acl authentication for proxy users and acl cahce reloading operations, like grant proxy.

With some debugging on top of MySQL server 8.0.2X, I believe the issue is from below piece of code:

sql/auth/sql_authentication.cc::acl_authenticate
......
3457       ACL_PROXY_USER *proxy_user;                                                <=== proxy user matching is done within acl_cache_lock scope
3458       /* check if the user is allowed to proxy as another user */
3459       Acl_cache_lock_guard acl_cache_lock(thd, Acl_cache_lock_mode::READ_MODE);
3460       if (!acl_cache_lock.lock()) return 1;
3461
3462       proxy_user =                                                               <=== proxy user copied into mpvio but proxied_host is referred by pointer to acl_cache
3463           acl_find_proxy_user(auth_user, sctx->host().str, sctx->ip().str,
3464                               mpvio.auth_info.authenticated_as, &is_proxy_user);
3465       acl_cache_lock.unlock();
......
3492         /* we're proxying : find the proxy user definition */
3493         if (!acl_cache_lock.lock()) return 1;
3494         acl_proxy_user = find_acl_user(proxy_user->get_proxied_host()            <=== later it tries to find actual proxied user with acl_cache_lock re-issued, but the host parameter could be invalid.
3495                                            ? proxy_user->get_proxied_host()
3496                                            : "",
3497                                        mpvio.auth_info.authenticated_as, true);

This could lead to a user login failure for proxy users.

How to repeat:
Normally, it's not easy to reproduce. I am able to repeat it constantly with a script that creating user, grating proxy, dropping user concurrently. I.E.

# for ((i=0; i<50; i++)); do ./test $i &  done

test.cc:
int main(int argc, const char *argv[]) {
  string user = "test";
  user.append(argv[1]);
  cout << "info:" << user << endl;
  string create_user_sql = "create USER " + user + " IDENTIFIED with mysql_native_password BY \"test\";";
  string drop_user_sql = "drop USER " + user + ";";
  string grant_user_sql = "grant proxy on root to " + user + ";";

  MYSQL *root_conn = mysql_init(NULL);
  if (!mysql_real_connect(root_conn, "127.0.0.1", "root", NULL, NULL, 3306, NULL, 0)) {
    goto out3;
  }

  if (mysql_query(root_conn, create_user_sql.c_str())) {  
    goto out3;
  }

  if (!mysql_query(root_conn, grant_user_sql.c_str())) {
    goto out2;
  }

  MYSQL *curr_conn = mysql_init(NULL);
  if (!mysql_real_connect(curr_conn, "127.0.0.1", user.c_str(), "test", NULL, 3306, NULL, 0)) {
    assert(0);
  }

  mysql_close(curr_conn);
out2:
  (void)mysql_query(root_conn, drop_user_sql.c_str());  
out3:
  mysql_close(root_conn);
  return 0;
}

Suggested fix:
Maybe rearrange a little bit on the acl_cache_lock scope or the find_acl_user timing。
[1 Nov 2022 13:28] MySQL Verification Team
Hi Mr. shang,

Thank you for your bug report.

However, we are not able to repeat the behaviour.

We have ran your C source code on 8.0.31 and 8.0.32 (not yet out) and we have not hit any race condition.

Also, your test case does not contain user names and their ACL's , including the one on the proxy user. If there is anything else that needs to be used, we have to be informed.

If you supply us with all necessary info for 8.0.31, we shall change the status to "Open".

Can't repeat.
[2 Nov 2022 16:01] jingliang shang
Hi Team,

Thanks for the effort. Maybe you can try below steps one more time, hopefully.
I am able to repeat this issue with these on mysql-server in 8.0.27 official docker.

step 0: 
Start up mysql-server docker and change root@localhost passwd to test, then open 2 console with docker exec -it ID bash

step1:
From console 1, run these commands which should never stop.
#mysql -uroot -ptest -hlocalhost --execute="set persist check_proxy_users=ON;"
#mysql -uroot -ptest -hlocalhost --execute="set persist mysql_native_password_proxy_users=ON;"
#mysql -uroot -ptest -hlocalhost --execute='create user test_proxy@localhost identified with mysql_native_password by "test"; grant proxy on root@localhost to test_proxy@localhost;'
#while true; do mysql -utest_proxy -ptest -hlocalhost --execute='select current_user;' || exit 1; done

step2:
From console 2, run this command until console 1 stops and return "ERROR 1045 (28000): Access denied for user 'test_proxy'@'localhost' (using password: YES)"
#while true; do mysql -uroot -hlocalhost -ptest --execute='create user a@a; grant proxy on root to a@a; revoke proxy on root from a@a; drop user a@a'; sleep 1& done

These script is not as efficient as the c code, but it should trigger issue within minutes.

Regards,
Shangj
[3 Nov 2022 13:02] MySQL Verification Team
Hi Mr. shang,

First of all , we test standalone MySQL and it's clients, not the ones running in any container.

We have tested your test case on 8.0.31 and the syntax that you used is already deprecated.

Can't repeat.