SVG stands for “Scalable Vector Graphics”. SVG defines vector-based graphics in XML format. Every element and every attribute in SVG files can be animated. SVG resource in lightning bundle can be used to define custom icons for your component. Here we are speaking about the icons and not logo. Icon and logo can look similar just like my blog’s icon and logo. But both are different. SVG deals with icons. By default if you see any component in salesforce, you can observe that default icon is .
If you go to edit page section or lightning page builder section and go to custom component section.

You can observe the default icon.
You can build a custom icon in SVG section of Lightning Bundle.
<?xml version="1.0"?>
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN"
"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
<svg xmlns="http://www.w3.org/2000/svg"
width="400" height="400">
<circle cx="100" cy="100" r="50" stroke="black"
stroke-width="5" fill="red" />
</svg>
The above design will create a icon just like this.

You can also refer to w3schools SVG Tutorial to build your own icon.
Advantage of using SVG.
- SVG images can be created and edited with any text editor
- SVG images can be searched, indexed, scripted, and compressed
- SVG images are scalable
- SVG images can be printed with high quality at any resolution
- SVG images are zoomable
- SVG graphics do NOT lose any quality if they are zoomed or resized
- SVG is an open standard
- SVG files are pure XML
Note:
SVG resource is save as componentName.svg. if your component name is My_Lightning_Component, then svg component name will be My_Lightning_Component.svg.