Judge Environment | Testlify, Inc. Help Center

Legend	Explanation
NZEC	Non Zero Exit Code
SIGSEGV	Segmentation Fault
SIGFPE	Floating Point Error
SIGABRT	Fatal Error
SIGXFSZ	Output is too large
TLE	Time Limit Exceeded
MLE	Memory Limit Exceeded
RE	Runtime Error
CE	Compilation Error

1. How does Testlify’s Online Judge determine whether the solution is correct?

Your code is evaluated automatically by a code-checker, not a human reviewer, so you need to write your code with this in mind. For each problem, there will be one or more input files matching the specifications provided in the problem statement, along with the corresponding correct output files. Your program is executed using these input files, and the generated output must match the correct output exactly to be judged as correct.

For example, if your program begins by printing “Enter the number” without the problem explicitly instructing you to do so, this extra output will cause your solution to fail, no matter what follows. The key is that the outputs must match exactly. It doesn’t matter at which point your program writes the correct output, as long as it does so correctly.

You do not need to read all input before starting to produce output. Instead, it’s common to print results as you process the input. Additionally, only standard input and output streams should be used for reading and writing; any other methods (e.g., command-line arguments, file I/O, dialog boxes) will result in a wrong submission.

2. How do I test my program on my local machine?

To test your program in the same way Testlify’s online judge evaluates it, create an input file and use the command line to redirect streams. For example:

./a.out < input > output

Here, your program reads input from the file named input and writes output to a file named output. You can then verify if your output matches the expected format specified in the problem.

3. How does the time limit work?

Your program must process all input files, including multiple test cases, within the specified time limit. For example, if the time limit is 1 second and there are 100 test cases in a single input file, your program must complete processing all 100 cases within 1 second, not 1 second per test case.

Some programming languages inherently run slower, so Testlify provides additional time for these (e.g., Java has a multiplier for the time limit). Regardless of the language, all input files are processed, even if the first input file fails.

4. How does the total execution time work?

The execution time displayed is the sum of the time taken to process all input files. However, if your program exceeds the specified time limit for any input file, it will be terminated immediately. Therefore, the total execution time is always less than or equal to the time limit multiplied by the number of input files.

5. How is memory usage determined?

The total memory consumed by your program includes memory allocated in the stack, data segment, heap, and BSS. For languages like Java, JavaScript, or Scala, the runtime environment’s initialization may lead to higher memory usage being displayed. While Testlify generally does not enforce strict memory limits for such languages, extreme memory usage may lead to your program being terminated.

6. Why doesn't my program compile?

C/C++: Ensure you’re using a standards-compliant compiler. Avoid using outdated compilers like Turbo C++. Remove headers like conio.h, which are incompatible.
Java: Make sure inner classes are static where required, as non-static classes may result in errors. Testlify will relax this constraint in the future.
Other languages: Refer to the specific compilation error for guidance.

7. Does TLE mean my code is correct but slow?

Not necessarily. Time Limit Exceeded (TLE) means your program failed to complete execution within the specified time limit and was forcibly stopped. This could be due to inefficient algorithms or infinite loops.

For example, if the limit is 2 seconds, a program that takes 2.001 seconds to execute will still be judged as exceeding the limit.

8. Why do I get Time Limit Exceeded (TLE)?

The most common cause of TLE is that your program is too slow. Optimize your code by selecting efficient algorithms, especially for problems with high constraints. Avoid nested loops where possible.

Input/output methods can also impact performance. For example:

In Java, use BufferedReader and BufferedWriter instead of Scanner and System.out.println.
In C++, use scanf and printf instead of cin and cout.

Remember that Testlify’s evaluation servers may run slower than your local machine, so solutions that are borderline on your system may fail online.

9. What is a Wrong Answer?

This means your program executed successfully but produced an incorrect result. Even if it passes the sample cases, additional test cases are used to validate your solution. Revisit the problem statement carefully to ensure correct interpretation.

10. Why do I get runtime errors?

A runtime error occurs when your program crashes or encounters unexpected behavior during execution. Common errors include:

SIGSEGV: Segmentation fault due to invalid memory access (e.g., accessing an array out of bounds).
SIGFPE: Division by zero or invalid floating-point operations.
SIGABRT: Program aborted due to insufficient memory or manual aborts.
NZEC: Non-zero exit code, often due to uncaught exceptions or missing return 0 in C/C++.
MLE: Memory Limit Exceeded, typically caused by overly large arrays or inefficient data structures.

To avoid runtime errors, validate all inputs, initialize variables properly, and check array bounds.

11. Can I view others' solutions?

No, Testlify maintains confidentiality. Similarly, no one else can view your solution.

12. Why is my submission queued?

High server traffic may cause slight delays in processing. If the system is under maintenance, submissions will be evaluated automatically once service resumes.

13. What should I do if submissions are not evaluated?

If your submissions are stuck or not evaluated, contact Testlify support at [email protected].

Comments and Queries

For clarifications or issues regarding test data or problem specifications, use the comment section on the problem’s page. Avoid sharing source code or spamming.

If your queries remain unresolved, email [email protected] for assistance.