[Lista ArNOG] Mikrotik CCR1072 - Buena experiencia (Edwin Salazar)
Jose Luis Gaspoz
gaspozj en is.com.ar
Mar Nov 29 10:17:22 ART 2016
Edwin:
Fíjate que en tu caso esta disponiendo de pocos servicios y sin embargo
tenes la CPU 10 al 65% !!!! ... eso si se mantiene un tiempo indica que no
es normal en la distribución de carga de procesos.
Saludos
Ing. Jose Luis Gaspoz
Internet Services S.A.
Tel: 0342-4565118
Cel: 342-5008523
-----Mensaje original-----
From: Edwin Salazar
Sent: Tuesday, November 29, 2016 9:34 AM
To: lista en arnog.com.ar
Subject: [Lista ArNOG] Mikrotik CCR1072 - Buena experiencia (Edwin Salazar)
Amigos,
En nuestro caso tenemos una implementación con ciertas diferencias al
escenario planteado pero no tenemos los resultados indicados, el router se
encuentra haciendo lo siguiente:
- Router de borde BGP
- IPv4-only
- BGP (1 carrier, GGC, NAP.EC)
- ruteo estático para la red interna
- Disbale Connection Tracking,
- NO Firewall, Mangle, NAT, DHCP, DNS, NAT
- No QoS
- IP services: ssh, winbox, www
[#####@Borde - BGP] > sys resource cpu print
# CPU LOAD IRQ
DISK
0 cpu0 0% 0%
0%
1 cpu1 1% 1%
0%
2 cpu2 2% 2%
0%
3 cpu3 0% 0%
0%
4 cpu4 0% 0%
0%
5 cpu5 1% 1%
0%
6 cpu6 0% 0%
0%
7 cpu7 0% 0%
0%
8 cpu8 16% 16%
0%
9 cpu9 4% 4%
0%
10 cpu10 65% 15%
0%
11 cpu11 0% 0%
0%
12 cpu12 0% 0%
0%
13 cpu13 0% 0%
0%
14 cpu14 4% 4%
0%
15 cpu15 0% 0%
0%
16 cpu16 1% 1%
0%
17 cpu17 0% 0%
0%
18 cpu18 1% 1%
0%
19 cpu19 4% 4%
0%
20 cpu20 0% 0%
0%
21 cpu21 0% 0%
0%
22 cpu22 0% 0%
0%
23 cpu23 1% 1%
0%
24 cpu24 2% 2%
0%
25 cpu25 4% 4%
0%
26 cpu26 0% 0%
0%
27 cpu27 2% 1%
0%
28 cpu28 3% 3%
0%
29 cpu29 0% 0%
0%
30 cpu30 1% 1%
0%
31 cpu31 1% 1%
0%
32 cpu32 5% 5%
0%
33 cpu33 1% 1%
0%
34 cpu34 0% 0%
0%
35 cpu35 2% 1%
0%
36 cpu36 0% 0%
0%
37 cpu37 2% 2%
0%
38 cpu38 1% 1%
0%
39 cpu39 3% 3%
0%
40 cpu40 0% 0%
0%
41 cpu41 1% 1%
0%
42 cpu42 13% 13%
0%
43 cpu43 7% 7%
0%
44 cpu44 0% 0%
0%
45 cpu45 2% 2%
0%
46 cpu46 0% 0%
0%
47 cpu47 4% 4%
0%
48 cpu48 0% 0%
0%
49 cpu49 7% 7%
0%
50 cpu50 0% 0%
0%
51 cpu51 12% 12%
0%
52 cpu52 4% 4%
0%
53 cpu53 2% 2%
0%
54 cpu54 0% 0%
0%
55 cpu55 0% 0%
0%
56 cpu56 11% 11%
0%
57 cpu57 0% 0%
0%
58 cpu58 17% 13%
0%
59 cpu59 2% 2%
0%
60 cpu60 2% 2%
0%
61 cpu61 1% 1%
0%
62 cpu62 0% 0%
0%
63 cpu63 1% 1%
0%
64 cpu64 5% 5%
0%
65 cpu65 2% 2%
0%
66 cpu66 2% 2%
0%
67 cpu67 1% 1%
0%
68 cpu68 0% 0%
0%
69 cpu69 0% 0%
0%
70 cpu70 6% 6%
0%
71 cpu71 0% 0%
0%
[####@Borde - BGP] > inter monitor-traffic sfp6-LAN
name: sfp6-LAN
rx-packets-per-second: 215 812
rx-bits-per-second: 908.3Mbps
fp-rx-packets-per-second: 215 812
fp-rx-bits-per-second: 908.3Mbps
rx-drops-per-second: 0
rx-errors-per-second: 0
tx-packets-per-second: 264 707
tx-bits-per-second: 2.0Gbps
fp-tx-packets-per-second: 264 707
fp-tx-bits-per-second: 2.0Gbps
tx-drops-per-second: 0
tx-errors-per-second: 0
-- [Q quit|D dump|C-z pause]
Probablemente puede ser un problema con algunos modelos, en nuestro caso
tenemos buenos resultados.
>>> ??
>>> Buenas,
>>> comparto una mala experiencia que tuvimos con el primer despliegue de un
>>> CCR1072 como edge-router. La intención es evaluar con otros operadores
>>> si
>>> es reversible, así como también que sirva de antecedente para otros
>>> tentados por los 72 cores o la cantidad de SFP+.
>>>
>>> Como RouterOS es un all-in-one es muy difícil comparar experiencias de
>>> otros dado que la combinatoria de features / protocolos que pueden estar
>>> activos es enorme. Por esto aclaro primero que nada el entorno de uso
>>> del
>>> equipo (y me atrevo a decir que seria el recomendado para todo router -y
>>> solo router- de frontera):
>>>
>>> - Router de "borde" de un ISP.
>>> - IPv4-only (ni siquiera el package ipv6 enabled).
>>> - eBGP hacia dos carriers, aceptando e instalando solo default-gw.
>>> - iBGP a un segundo borde para redundancia active:active.
>>> - routing estático hacia adentro (no IGP, se podría decir que el único
>>> protocolo de routing activo es BGP).
>>> - Firewalling: sin estado (ninguna regla tiene connection-state como
>>> matcheo), sin reglas de Mangle, sin reglas de NAT,
>>> "connection-tracking=no".
>>> - No se brinda ningún servicio a la red (DNS, DHCP, NAT, etc) desde el
>>> router.
>>> - QoS: nada de nada, vacío de config el apartado queue.
>>> - Desactivado temporal por típica "a ver si es esto" en RouterOS: SNMP,
>>> todos los ip services menos winbox y ssh, logging.
>>>
>>>
>>> ## Maniobras realizadas a pedido del soporte Mikrotik que no
>>> tuvieron éxito (a pesar de que algunas rompen la necesidad o la idea de
>>> un
>>> equipo en producción original):
>>> - pasar por todas las last-version de cada rama de RouterOS (bugfix,
>>> current, RC).
>>> - disable de todas la reglas de firewall (que no hacían uso de
>>> connection-tracking).
>>> - usar varias interfaces SFP+ físicas individuales en vez de trunkear
>>> todo en vlans en unas pocas (ej 10G-WANs , 10G-CORE).
>>> - actualizar el firmware del RouterBoard.
>>>
>>> ## Detalle *MUY IMPORTANTE:*
>>> - El CCR1072 se desplegó como remplazo natural del CCR1036 que estaba (y
>>> dado esto, sigue) en producción, pensando mayormente en eliminar los
>>> LAGs
>>> Nx1G y simplificar la conexión física a los carriers que se supera el
>>> Gbps
>>> de trafico contratado.
>>> - Por esta razón, en *cuanto a config el CCR1072 es un export/import
>>> literal de la config del CCR1036. Si hacemos un diff entre uno y otro
>>> solo varían las lineas referidas a las interfaces físicas* (incluso
>>> tratamos de imitar el uso de un LAG innecesario en el CCR1072 apostando
>>> a
>>> un tema de interrupciones).
>>> - Con esto quiero remarcar que a exactamente igual config y mismo
>>> tráfico
>>> agregado (equipo de remplazo) que el CCR1036, *el uso de CPU es horrible
>>> en el CCR1072*. Mal distribuido principalmente (haciendo uso de no mas
>>> de 4 o 5 cores) y generando peaks en cores individuales, combo que
>>> reduce
>>> enormemente la escalabilidad del equipo.
>>>
>>>
>>>
>>>
>>>
>>> ?
>>>
>>>
>>>
>>>
>>> ### Analizando con profile tool, en muchas ocasiones *"unclassified"*
>>> proccess es el que se lleva gran parte de los cores que se pican
>>> intercaladamente a valores altos con respecto a la carga de tráfico:
>>>
>>>
>>> CCR1072-BORDE] > tool profile cpu=2
>>> NAME CPU USAGE
>>> firewall 2 0%
>>> networking 2 1%
>>> management 2 0%
>>> routing 2 0%
>>> idle 2 82%
>>> profiling 2 1%
>>> *unclassified 2 16% **<<< WTF!*
>>>
>>>
>>> CCR1072-BORDE] > tool profile cpu=2
>>> NAME CPU USAGE
>>> firewall 2 1%
>>> networking 2 0%
>>> management 2 0%
>>> routing 2 0%
>>> idle 2 65%
>>> profiling 2 0%
>>> *unclassified 2 34% <<< WTF!*
>>>
>>>
>>> ### Incluso en horarios de poco trafico:
>>>
>>> CCR1072-BORDE] > interface monitor-traffic sfp-sfpplus5
>>> name: sfp-sfpplus5
>>> rx-packets-per-second: 40 490
>>> rx-bits-per-second: 387.9Mbps
>>> fp-rx-packets-per-second: 40 490
>>> *fp-rx-bits-per-second: 387.9Mbps*
>>> rx-drops-per-second: 0
>>> rx-errors-per-second: 0
>>> tx-packets-per-second: 40 259
>>> tx-bits-per-second: 387.6Mbps
>>> fp-tx-packets-per-second: 40 259
>>> *fp-tx-bits-per-second: 387.6Mbps*
>>> tx-drops-per-second: 0
>>> tx-errors-per-second: 0
>>>
>>> ### Como se puede ver 60 o mas cores duermen la siesta mientras un par
>>> hacen todo el trabajo, en los 1072 reina el sindicalismo :P
>>>
>>>
>>> CCR1072-BORDE] > system resource cpu print
>>> # CPU
>>>
>>> LOAD
>>> IRQ DISK
>>> 0 cpu0
>>>
>>> 0%
>>> 0% 0%
>>> 1 cpu1
>>>
>>> 0%
>>> 0% 0%
>>> 2 cpu2
>>>
>>> 0%
>>> 0% 0%
>>> 3 cpu3
>>>
>>> 0%
>>> 0% 0%
>>> 4 cpu4
>>> *
>>> 0% 0% 0%*
>>> * 5 cpu5
>>>
>>> 37% 1% 0%*
>>> 6 cpu6
>>>
>>> 0%
>>> 0% 0%
>>> 7 cpu7
>>>
>>> 0%
>>> 0% 0%
>>> 8 cpu8
>>>
>>> 0%
>>> 0% 0%
>>> 9 cpu9
>>>
>>> 0%
>>> 0% 0%
>>> 10 cpu10
>>>
>>> 0%
>>> 0% 0%
>>> 11 cpu11
>>>
>>> 0%
>>> 0% 0%
>>> 12 cpu12
>>>
>>> 0%
>>> 0% 0%
>>> 13 cpu13
>>>
>>> 0%
>>> 0% 0%
>>> 14 cpu14
>>>
>>> 0%
>>> 0% 0%
>>> 15 cpu15
>>>
>>> 0%
>>> 0% 0%
>>> 16 cpu16
>>>
>>> 0%
>>> 0% 0%
>>> 17 cpu17
>>>
>>> 6%
>>> 6% 0%
>>> 18 cpu18
>>>
>>> 0%
>>> 0% 0%
>>> 19 cpu19
>>>
>>> 0%
>>> 0% 0%
>>> 20 cpu20
>>>
>>> 0%
>>> 0% 0%
>>> 21 cpu21
>>>
>>> 0%
>>> 0% 0%
>>> 22 cpu22
>>>
>>> 0%
>>> 0% 0%
>>> 23 cpu23
>>>
>>> 0%
>>> 0% 0%
>>> 24 cpu24
>>>
>>> 0%
>>> 0% 0%
>>> 25 cpu25
>>>
>>> 0%
>>> 0% 0%
>>> 26 cpu26
>>>
>>> 0%
>>> 0% 0%
>>> 27 cpu27
>>>
>>> 0%
>>> 0% 0%
>>> 28 cpu28
>>>
>>> 0%
>>> 0% 0%
>>> 29 cpu29
>>>
>>> 0%
>>> 0% 0%
>>> 30 cpu30
>>>
>>> 0%
>>> 0% 0%
>>> 31 cpu31
>>>
>>> 0%
>>> 0% 0%
>>> 32 cpu32
>>>
>>> 0%
>>> 0% 0%
>>> 33 cpu33
>>>
>>> 0%
>>> 0% 0%
>>> 34 cpu34
>>>
>>> 0%
>>> 0% 0%
>>> 35 cpu35
>>>
>>> 2%
>>> 2% 0%
>>> 36 cpu36
>>>
>>> 0%
>>> 0% 0%
>>> *37 cpu37
>>>
>>> 34%
>>> 33% 0%*
>>> 38 cpu38
>>>
>>> 0%
>>> 0% 0%
>>> 39 cpu39
>>>
>>> 2%
>>> 2% 0%
>>> 40 cpu40
>>>
>>> 0%
>>> 0% 0%
>>> 41 cpu41
>>>
>>> 0%
>>> 0% 0%
>>> 42 cpu42
>>>
>>> 0%
>>> 0% 0%
>>> 43 cpu43
>>>
>>> 0%
>>> 0% 0%
>>> 44 cpu44
>>>
>>> 0%
>>> 0% 0%
>>> 45 cpu45
>>>
>>> 0%
>>> 0% 0%
>>> 46 cpu46
>>>
>>> 0%
>>> 0% 0%
>>> 47 cpu47
>>>
>>> 1%
>>> 1% 0%
>>> 48 cpu48
>>>
>>> 0%
>>> 0% 0%
>>> 49 cpu49
>>>
>>> 0%
>>> 0% 0%
>>> 50 cpu50
>>>
>>> 0%
>>> 0% 0%
>>> 51 cpu51
>>>
>>> 0%
>>> 0% 0%
>>> 52 cpu52
>>>
>>> 0%
>>> 0% 0%
>>> 53 cpu53
>>>
>>> 0%
>>> 0% 0%
>>> 54 cpu54
>>>
>>> 0%
>>> 0% 0%
>>> 55 cpu55
>>>
>>> 0%
>>> 0% 0%
>>> 56 cpu56
>>>
>>> 0%
>>> 0% 0%
>>> 57 cpu57
>>>
>>> 0%
>>> 0% 0%
>>> 58 cpu58
>>>
>>> 0%
>>> 0% 0%
>>> 59 cpu59
>>>
>>> 0%
>>> 0% 0%
>>> 60 cpu60
>>>
>>> 0%
>>> 0% 0%
>>> 61 cpu61
>>>
>>> 0%
>>> 0% 0%
>>> 62 cpu62
>>>
>>> 0%
>>> 0% 0%
>>> 63 cpu63
>>>
>>> 0%
>>> 0% 0%
>>> 64 cpu64
>>>
>>> 3%
>>> 2% 0%
>>> 65 cpu65
>>>
>>> 0%
>>> 0% 0%
>>> 66 cpu66
>>>
>>> 1%
>>> 0% 0%
>>> 67 cpu67
>>>
>>> 0%
>>> 0% 0%
>>> 68 cpu68
>>>
>>> 0%
>>> 0% 0%
>>> 69 cpu69
>>>
>>> 0%
>>> 0% 0%
>>> 70 cpu70
>>>
>>> 0%
>>> 0% 0%
>>> 71 cpu71
>>>
>>> 0%
>>> 0% 0%
>>>
>>>
>>> --
>>>
>>> *Ivan ChaperoÁrea Técnica y Soporte*
>>> Fijo: 03464-470280 (interno 535) | Móvil: 03464-155-20282 | Skype ID:
>>> ivanchapero
>>> --
>>> GoDATA Banda Ancha - CABLETEL S.A. | Av. 9 de Julio 1163 - 2183 -
>>> Arequito - Santa Fe - Argentina
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> Lista mailing list
>>> Lista en arnog.com.ar
>>> http://mailmancabase.interdotnet.com.ar/mailman/listinfo/lista
>>>
>>>
>>
>> _______________________________________________
>> Lista mailing list
>> Lista en arnog.com.ar
>> http://mailmancabase.interdotnet.com.ar/mailman/listinfo/lista
>>
>>
>> _______________________________________________
>> Lista mailing list
>> Lista en arnog.com.ar
>> http://mailmancabase.interdotnet.com.ar/mailman/listinfo/lista
>>
>>
> ------------ próxima parte ------------
> Se ha borrado un adjunto en formato HTML...
> URL:
> <http://mailmancabase.interdotnet.com.ar/pipermail/lista/attachments/20161127/86499f69/attachment.html>
>
> ------------------------------
>
> _______________________________________________
> Lista mailing list
> Lista en arnog.com.ar
> http://mailmancabase.interdotnet.com.ar/mailman/listinfo/lista
>
>
> Fin de Resumen de Lista, Vol 114, Envío 8
> *****************************************
_______________________________________________
Lista mailing list
Lista en arnog.com.ar
http://mailmancabase.interdotnet.com.ar/mailman/listinfo/lista
-----
Se certificó que el correo no contiene virus.
Comprobada por AVG - www.avg.com
Versión: 2016.0.7924 / Base de datos de virus: 4728/13500 - Fecha de la
versión: 29/11/2016
Más información sobre la lista de distribución Lista