-
Notifications
You must be signed in to change notification settings - Fork 3.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[cherry-pick](branch-2.1) pick hive text write from master #40537
[cherry-pick](branch-2.1) pick hive text write from master #40537
Conversation
Thank you for your contribution to Apache Doris. Since 2024-03-18, the Document has been moved to doris-website. |
run buildall |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
clang-tidy made some suggestions
run buildall |
1. Support write hive text table 2. Add SessionVariable `hive_text_compression` to write compressed hive text table 3. Supported compression type: gzip, bzip2, snappy, lz4, zstd
## abstract introduces the pluginization of the Authenticator component, enabling greater flexibility and modularity within the authentication system. By refactoring the Authenticator into a pluggable SPI (Service Provider Interface), we allow for custom authentication mechanisms to be seamlessly integrated and managed. ## Key Changes AuthenticatorFactory Interface: This interface defines the create method, which is used to create an Authenticator instance based on initialization properties, enabling support for various authentication mechanisms. The factoryIdentifier method provides an identifier for the factory, allowing differentiation between various AuthenticatorFactory implementations (e.g., LDAP, default). Implemented Specific Authentication Factories: We have implemented factories for different authentication mechanisms (such as LDAP), providing the necessary configuration support for each. ## eg In your Maven pom.xml, add the fe-core dependency: ``` <dependency> <groupId>org.apache.doris</groupId> <artifactId>fe-core</artifactId> <version>1.2-SNAPSHOT</version> <scope>provided</scope> </dependency> ``` ``` import org.apache.doris.common.ConfigBase; public class LocalAuthenticatorFactory implements AuthenticatorFactory { @OverRide public LocalAuthenticator create(Properties initProps) { return new LocalAuthenticator(); } @OverRide public String factoryIdentifier() { return "local"; } } import java.util.HashMap; import java.util.Map; public class LocalAuthenticator implements Authenticator { private Map<String, String> userStore = new HashMap<>(); public LocalAuthenticator() { // Hardcoded usernames and passwords userStore.put("user1", "password1"); userStore.put("user2", "password2"); userStore.put("admin", "adminpass"); } @OverRide public boolean authenticate(String username, String password) { // Authenticate by checking the hardcoded user store String storedPassword = userStore.get(username); return storedPassword != null && storedPassword.equals(password); } } ``` To integrate with the SPI (Service Provider Interface) mechanism, we created a file at **src/main/resources/META-INF/services/org.apache.doris.mysql.authenticate.AuthenticatorFactory** containing **org.example.LocalAuthenticatorFactory,** which allows the system to dynamically load and utilize the LocalAuthenticatorFactory implementation. Your plugin needs to include all necessary dependencies, and it is generally recommended to package them into an Uber JAR (also known as a Fat JAR) to ensure that the plugin can run independently without additional dependency management. Once packaged, this JAR can be placed directly in the **fe/custom_lib** directory.
Impl ZstdDecompressor and support read hive text table which is compressed by zstd.
fe4ef6e
to
b95fdce
Compare
run buildall |
TeamCity be ut coverage result: |
run p0 |
run buildall |
TeamCity be ut coverage result: |
run buildall |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
PR approved by at least one committer and no changes requested. |
PR approved by anyone and no changes requested. |
TeamCity be ut coverage result: |
run p0 |
Proposed changes
pick prs:
#38549
#40183
#40315