Thanks a lot for your reply Marc. I browsed through the older threads and didn't find a good solution for this.
"BigTable cannot keep up with your index repair workers" - could you provide a little bit insights for what an index repair job does, or any documentation?
I was trying a few storage settings and didn't get any luck yet: storage.write-time/storage.lock.wait-time/storage.lock.expiry-time/etc. Do you think it will make a difference?
As you suggested, I'll try delete the index and retry from start.
For our application, we do need to have the option of reindexing current data, so I'll need to make sure it works. Do you see similar issue for Cassandra? We deploy it on GCP so we try Bigtable first.
Do you have any recommendation on backend storage for GCP please?