Hi Felix,
You can modify (delete) the board outline by selecting the Board Outline layer in the Layers tab in the Design Toolbox. Note that the Board Outline layer is selected in the image below. Once you select the layer, you will be able to select items (lines, rectangle, etc) on that layer.