Skip to content

Commit 09dafe2

Browse files
Remove runtime stderr warning from Python bindings (#1898)
* Initial plan * Remove runtime stderr warning from Python bindings Remove the eprintln! calls that printed a runtime warning to stderr when the process was forked after parallelism had been used. The set_parallelism(false) behavior is preserved to avoid deadlocks. File changed: bindings/python/src/lib.rs Commit OID referenced: a05b60c Co-authored-by: ArthurZucker <[email protected]> * Update comment to reflect the new behavior after removing warning The comment now correctly states that the callback disables parallelism instead of warning the user. Co-authored-by: ArthurZucker <[email protected]> --------- Co-authored-by: copilot-swe-agent[bot] <[email protected]> Co-authored-by: ArthurZucker <[email protected]>
1 parent a05b60c commit 09dafe2

File tree

1 file changed

+1
-10
lines changed

1 file changed

+1
-10
lines changed

bindings/python/src/lib.rs

Lines changed: 1 addition & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -37,22 +37,13 @@ pub const VERSION: &str = env!("CARGO_PKG_VERSION");
3737

3838
// For users using multiprocessing in python, it is quite easy to fork the process running
3939
// tokenizers, ending up with a deadlock because we internally make use of multithreading. So
40-
// we register a callback to be called in the event of a fork so that we can warn the user.
40+
// we register a callback to be called in the event of a fork to disable parallelism.
4141
#[cfg(target_family = "unix")]
4242
static mut REGISTERED_FORK_CALLBACK: bool = false;
4343
#[cfg(target_family = "unix")]
4444
extern "C" fn child_after_fork() {
4545
use tk::parallelism::*;
4646
if has_parallelism_been_used() && !is_parallelism_configured() {
47-
eprintln!(
48-
"huggingface/tokenizers: The current process just got forked, after parallelism has \
49-
already been used. Disabling parallelism to avoid deadlocks..."
50-
);
51-
eprintln!("To disable this warning, you can either:");
52-
eprintln!(
53-
"\t- Avoid using `tokenizers` before the fork if possible\n\
54-
\t- Explicitly set the environment variable {ENV_VARIABLE}=(true | false)"
55-
);
5647
set_parallelism(false);
5748
}
5849
}

0 commit comments

Comments
 (0)