BGP: Allow operator to choose whether MED is compared for paths learned from different neighboring ASNs

[taspelund]

The current implementation of BGP bestpath always paths by neighboring ASN before comparing MED (per the BGP-4 RFC).
However, this prevents a BGP router from being able to consider two paths from different neighboring ASNs as multipath candidates at the same time.

It would be good to update our bestpath behavior to accommodate multipath across different neighboring ASNs, as that BGP design is not uncommon in enterprise DC deployments.

[taspelund]

I discussed this a bit more with @rcgoodfellow. After stepping back and talking through the problem space, I think what I'm actually wanting with this ticket is some configurability of the bestpath algorithm.

Specifically, I see maghemite being used in two different deployment styles:

1. No-trust (traditional Service Provider style deployments where other ASNs are separate/untrusted administrative domains)
2. Trusted (Datacenter style deployments where the local BGP router and other ASNs are all part of the same administrative domain)

These two deployment models often have different requirements in terms of how "easy" it is for certain things to happen, e.g. multipath and use of MED to influence bestpath.

In the No-trust style deployment, you generally don't want the MED on a path from ASN X to influence the likelihood of selecting the path from ASN Y as the bestpath. Additionally, you may or may not want to enable multipathing across discrete ASNs (e.g. accepting a default from both ASN X and ASN Y, then installing that route into the RIB with two ECMP paths) depending on whether or not you have a greater trust/prefererence for one of those paths.

Whereas in the Trusted style deployment, the preference for a path would normally be based on metric (MED) rather than by ASN. In these deployments it should be simple both to treat paths from various ASNs as valid for multipathing and to use MED to influence path selection despite coming from separate ASNs.

This means we likely will want the ability to modify both MED evaluation across ASNs (always-compare-med) and the ability to create ECMP groups comprised of paths from different neighboring ASNs (bgp bestpath aspath-relax).