Creating Small Competitions¶

If you go to the Competitions section in the control panel, you will have the option to create a small competition by clicking Create Competition. You will get a window where you can name the competition and set the start and end dates.

When you click Create, you will proceed to set the competition parameters. First, if you click on the Settings field below the competition name, you will be able to reset the name, description, start and end of the competition, but you will also have a number of additional options. You can choose the moment when the results will be frozen, which means that after that moment, competitors will no longer be able to see changes in the ranking list.

When you scroll down, the next thing you will see is the visibility settings. If you check that the competition is public, all Petlja users will be able to see it and all users will be able to solve the problems, but only those you add as participants will enter the ranking list (more about adding participants later in the guide). If the field Results visible to competitors is checked, only competitors will be able to see the ranking list, and if the field Results visible publicly is checked, all users will be able to see the ranking list. If neither of these two is checked, only you and participants marked as organizers will be able to see the ranking list. If the field Allow access to the environment after the end of the competition is checked, participants will be able to solve problems after the competition ends, but their solutions will not enter the ranking list. If you check the Archive field, the competition will no longer be visible in the Arena under My Competitions, which can be useful if you have a very large number of competitions and do not want this competition to remain visible in the Arena.

After that, you will have the option to set the scoring method. If the field Best result counts is checked, the best submitted solution for each problem will be counted. If this field is unchecked, the last submitted solution for each problem will be counted.

You also have the option to introduce penalties for the number of submitted solutions (the more attempts, the worse the result) and for the time elapsed since the start of the competition.

Finally, if the option Always check all test cases is checked, every submitted solution will be run against every test case. If this option is not checked, testing of the submitted solution stops when a test is encountered where the program does not give the expected answer within the given time and memory limits. This way of testing makes sense (reduces system load) if only completely correct problems are accepted.

At the end, you can set the minimum time between two submitted solutions, which will prevent competitors from submitting solutions at intervals shorter than the one you specified and thus “flooding” the system with testing requests. In the next field, you can also specify how many solutions a competitor can submit in total for each problem.

If the field Questions active is checked, in Petlja’s competition environment, competitors will be able to ask questions during the competition by clicking the Questions button. There they will also be able to see your answers to their questions. Also, if Notifications active is checked, you will have the option to publish notifications that competitors will see during the competition by clicking the Notifications button, which exists in Petlja’s competition environment. See this guide to see what this looks like from the competitor’s perspective. In the Materials field, you can enter the name of additional material that will be available to competitors during the competition, as well as a link to that material.

Problems¶

By clicking on the problems field, you will access the selection of programming languages available to competitors as well as the selection of problems. Simply check the programming languages you want to make available in the competition. You add problems by selecting a problem among Petlja’s materials and simply typing the name of the problem in the “Add problem” field. If you wish, you can change the name of each problem within your competition by entering the desired name in the “Name within competition” field. When you do this, click the plus sign and the problem will appear in your competition. You also have the option to remove a problem by clicking the trash can.

Scoring¶

If problems are simply added to the competition without specifically defining the scoring method, each test case of each problem will be worth 1 point. However, there is also the possibility to customize the scoring system to the needs of the competition owner. Below the problems section, there is a section intended for evaluation. On the left side, there is a dropdown menu where you need to enter the letter that marks the problem for which you want to customize the scoring.

In this case, the problem “Stolice” has the label A, and scoring will be set for it. Then you need to enter the settings in yaml format in the text editor, which will allow our system to know how to automatically evaluate the solutions. The settings will be explained in the following two examples.

First of all, you need to set type to one of two possible values: testcase or subtask. If type is set to testcase, scoring will be done separately for each test case and each test case can carry any number of points (you can see the test cases for each problem as explained in the “Collection” chapter of this guide). If type is set to subtask, test cases are grouped into subtasks and evaluation is done based on the results on all test cases within a subtask (group of test cases). Score_total represents the total number of points that the problem carries and must exist both when type is set to subtask and when type is set to testcase. Note that this value should match the sum of the points of all test cases or subtasks.

Groups is used exclusively when type is set to subtask and must contain 3 elements:

• id – identifies the subtask and the subtasks are ordered according to this element. In the example above, these are subtasks 1, 2, 3.

• score – the number of points this subtask carries if all tests in that subtask have status OK

• testcases – the ordinal numbers of the test cases included in this subtask (counting from 1).

In example 1 above, we can see that the selected problem is set to have 3 subtasks - Groups defines 3 subtasks, 1, 2, 3, of which 1 carries 20 points, 3 carries 50, and 2 carries 30 (defined by score). Subtask 1 includes test cases 1, 2, 3, subtask 2 includes test cases 4, 5, 6, and subtask 3 includes test cases 7, 8, 9, 10 (defined by testcases).

In example 2 above, the way to set the value of test cases if you choose that scoring method is shown. score_overrides is used exclusively when type is set to testcase and when you do not want a uniform distribution of points per test case. After that, you need to define the following elements:

• name: the ordinal number of the specific test case (counting from 1)

• score: the number of points that test case carries

Note that all test cases not listed in score_overrides carry an equal number of points, which is determined based on the score_total value. If the problem carries 100 points (defined by score_total) in total and there are 12 test cases, and for two test cases it is defined by score_overrides that they carry 0 points as in the example above, the remaining points are distributed uniformly among all other cases (each test case will carry 10 points). Here, make sure that the remaining number of points (the difference between the total and the sum of the explicitly set scores) is divisible by the number of remaining cases (otherwise you will get an error). Finally, public is a list that contains all the identifiers of test cases or subtasks that are public, i.e., for which the competitor receives full feedback (can see for which values their program gave the correct answer and for which it did not). It is common to set public tests to carry zero points, although any other setting is possible.

When you enter all the scoring details for a given problem, be sure to click the up arrow for uploading at the bottom of the scoring description input window (scroll down if necessary).

After clicking, wait to see if you get any error messages. Among other things, keep in mind that the YAML format is very sensitive to leading spaces in lines.

Participants¶

By clicking on the “Participants” field, you will be able to control who the participants of your competition are. In the User/Group field, you need to enter the username of the user or the name of the group you want to add. You assign a role to the user or group by selecting in the “Role” field whether it will be an organizer or a competitor. You can also assign a penalty to each user with which they will start the competition (if fifth and sixth grade students are competing, for example, it is possible to penalize sixth graders). You add participants by clicking the plus sign, and, if necessary, remove them by clicking the trash can.