[Lista ArNOG] Buena experiencia CCR1072 (Edwin Salazar)

Edwin Salazar edwin.salazar en ixp.ec
Mar Nov 29 11:09:34 ART 2016


Jose Luis, Ivan

En este caso sube esporádicamente y de forma aleatoria pero no se mantiene.

El trafico que estamos pasando no es el mismo, en tu caso pasan casi 400Mbps y nosotros 3.0Gbps.

Adjunto una captura adicional para que lo puedan ver.



 # CPU                                       LOAD         IRQ        DISK
 0 cpu0                                        0%          0%          0%
 1 cpu1                                        0%          0%          0%
 2 cpu2                                        0%          0%          0%
 3 cpu3                                        1%          1%          0%
 4 cpu4                                        3%          2%          0%
 5 cpu5                                        4%          4%          0%
 6 cpu6                                       10%         10%          0%
 7 cpu7                                        6%          6%          0%
 8 cpu8                                        1%          1%          0%
 9 cpu9                                        1%          1%          0%
10 cpu10                                       0%          0%          0%
11 cpu11                                       0%          0%          0%
12 cpu12                                       1%          1%          0%
13 cpu13                                      17%         16%          0%
14 cpu14                                      32%         32%          0%
15 cpu15                                       3%          3%          0%
16 cpu16                                       2%          2%          0%
17 cpu17                                       4%          4%          0%
18 cpu18                                       0%          0%          0%
19 cpu19                                       0%          0%          0%
20 cpu20                                      32%         32%          0%
21 cpu21                                      11%         10%          0%
22 cpu22                                       7%          7%          0%
23 cpu23                                       7%          7%          0%
24 cpu24                                       0%          0%          0%
25 cpu25                                       0%          0%          0%
26 cpu26                                       0%          0%          0%
27 cpu27                                       7%          7%          0%
28 cpu28                                       5%          5%          0%
29 cpu29                                      10%         10%          0%
30 cpu30                                      10%         10%          0%
31 cpu31                                       7%          7%          0%
32 cpu32                                      18%         17%          0%
33 cpu33                                       1%          1%          0%
34 cpu34                                       4%          4%          0%
35 cpu35                                      14%         14%          0%
36 cpu36                                      21%         21%          0%
37 cpu37                                       4%          4%          0%
38 cpu38                                      10%         10%          0%
39 cpu39                                      16%         16%          0%
40 cpu40                                       9%          9%          0%
41 cpu41                                      18%         18%          0%
42 cpu42                                      12%         12%          0%
43 cpu43                                      19%         19%          0%
44 cpu44                                      18%         18%          0%
45 cpu45                                      19%         19%          0%
46 cpu46                                      17%         17%          0%
47 cpu47                                      24%         22%          0%
48 cpu48                                       9%          9%          0%
49 cpu49                                      19%         19%          0%
50 cpu50                                      13%         13%          0%
51 cpu51                                      19%         19%          0%
52 cpu52                                      22%         20%          0%
53 cpu53                                      15%         15%          0%
54 cpu54                                      24%         24%          0%
55 cpu55                                      25%         25%          0%
56 cpu56                                       6%          6%          0%
57 cpu57                                      18%         18%          0%
58 cpu58                                       9%          9%          0%
59 cpu59                                       9%          9%          0%
60 cpu60                                      22%         22%          0%
61 cpu61                                      21%         21%          0%
62 cpu62                                      20%         20%          0%
63 cpu63                                      15%         15%          0%
64 cpu64                                      10%         10%          0%
65 cpu65                                      12%         12%          0%
66 cpu66                                      16%         16%          0%
67 cpu67                                      16%         16%          0%
68 cpu68                                      15%         15%          0%
69 cpu69                                      18%         18%          0%
70 cpu70                                      22%         22%          0%
71 cpu71                                       0%          0%          0%

Salu2.

> 
> ----------------------------------------------------------------------
> 
> Message: 1
> Date: Tue, 29 Nov 2016 10:47:43 -0300
> From: Ivan Chapero <info en ivanchapero.com.ar>
> To: lista en arnog.com.ar, Jose Luis Gaspoz <gaspozj en is.com.ar>
> Subject: Re: [Lista ArNOG] Mikrotik CCR1072 - Buena experiencia (Edwin
> 	Salazar)
> Message-ID:
> 	<CAPQhFbe2d+JT65LeGUhm6BAXotC4u76vEyTW+5b+UO58HDRBNQ en mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
> 
> Me sorprende que con firewall OFF, tengas aun mas carga que la foto de
> rendimiento que subi con el mismo trafico.
> 
> El 29 de noviembre de 2016, 10:17, Jose Luis Gaspoz <gaspozj en is.com.ar>
> escribió:
> 
>> Edwin:
>> 
>> Fíjate que en tu caso esta disponiendo de pocos servicios y sin embargo
>> tenes la CPU 10 al 65% !!!! ... eso si se mantiene un tiempo indica que no
>> es normal en la distribución de carga de procesos.
>> 
>> Saludos
>> 
>> Ing. Jose Luis Gaspoz
>> Internet Services S.A.
>> Tel: 0342-4565118
>> Cel: 342-5008523
>> -----Mensaje original----- From: Edwin Salazar
>> Sent: Tuesday, November 29, 2016 9:34 AM
>> To: lista en arnog.com.ar
>> Subject: [Lista ArNOG] Mikrotik CCR1072 - Buena experiencia (Edwin Salazar)
>> 
>> 
>> Amigos,
>> 
>> En nuestro caso tenemos una implementación con ciertas diferencias al
>> escenario planteado pero no tenemos los resultados indicados, el router se
>> encuentra haciendo lo siguiente:
>> 
>> - Router de borde BGP
>> - IPv4-only
>> - BGP (1 carrier, GGC, NAP.EC)
>> - ruteo estático para la red interna
>> - Disbale Connection Tracking,
>> - NO Firewall, Mangle, NAT, DHCP, DNS, NAT
>> - No QoS
>> - IP services: ssh, winbox, www
>> 
>> [#####@Borde - BGP] > sys resource cpu print
>> # CPU                                                LOAD         IRQ DISK
>> 0 cpu0                                                 0%          0% 0%
>> 1 cpu1                                                 1%          1% 0%
>> 2 cpu2                                                 2%          2% 0%
>> 3 cpu3                                                 0%          0% 0%
>> 4 cpu4                                                 0%          0% 0%
>> 5 cpu5                                                 1%          1% 0%
>> 6 cpu6                                                 0%          0% 0%
>> 7 cpu7                                                 0%          0% 0%
>> 8 cpu8                                                16%         16% 0%
>> 9 cpu9                                                 4%          4% 0%
>> 10 cpu10                                               65%         15% 0%
>> 11 cpu11                                                0%          0% 0%
>> 12 cpu12                                                0%          0% 0%
>> 13 cpu13                                                0%          0% 0%
>> 14 cpu14                                                4%          4% 0%
>> 15 cpu15                                                0%          0% 0%
>> 16 cpu16                                                1%          1% 0%
>> 17 cpu17                                                0%          0% 0%
>> 18 cpu18                                                1%          1% 0%
>> 19 cpu19                                                4%          4% 0%
>> 20 cpu20                                                0%          0% 0%
>> 21 cpu21                                                0%          0% 0%
>> 22 cpu22                                                0%          0% 0%
>> 23 cpu23                                                1%          1% 0%
>> 24 cpu24                                                2%          2% 0%
>> 25 cpu25                                                4%          4% 0%
>> 26 cpu26                                                0%          0% 0%
>> 27 cpu27                                                2%          1% 0%
>> 28 cpu28                                                3%          3% 0%
>> 29 cpu29                                                0%          0% 0%
>> 30 cpu30                                                1%          1% 0%
>> 31 cpu31                                                1%          1% 0%
>> 32 cpu32                                                5%          5% 0%
>> 33 cpu33                                                1%          1% 0%
>> 34 cpu34                                                0%          0% 0%
>> 35 cpu35                                                2%          1% 0%
>> 36 cpu36                                                0%          0% 0%
>> 37 cpu37                                                2%          2% 0%
>> 38 cpu38                                                1%          1% 0%
>> 39 cpu39                                                3%          3% 0%
>> 40 cpu40                                                0%          0% 0%
>> 41 cpu41                                                1%          1% 0%
>> 42 cpu42                                               13%         13% 0%
>> 43 cpu43                                                7%          7% 0%
>> 44 cpu44                                                0%          0% 0%
>> 45 cpu45                                                2%          2% 0%
>> 46 cpu46                                                0%          0% 0%
>> 47 cpu47                                                4%          4% 0%
>> 48 cpu48                                                0%          0% 0%
>> 49 cpu49                                                7%          7% 0%
>> 50 cpu50                                                0%          0% 0%
>> 51 cpu51                                               12%         12% 0%
>> 52 cpu52                                                4%          4% 0%
>> 53 cpu53                                                2%          2% 0%
>> 54 cpu54                                                0%          0% 0%
>> 55 cpu55                                                0%          0% 0%
>> 56 cpu56                                               11%         11% 0%
>> 57 cpu57                                                0%          0% 0%
>> 58 cpu58                                               17%         13% 0%
>> 59 cpu59                                                2%          2% 0%
>> 60 cpu60                                                2%          2% 0%
>> 61 cpu61                                                1%          1% 0%
>> 62 cpu62                                                0%          0% 0%
>> 63 cpu63                                                1%          1% 0%
>> 64 cpu64                                                5%          5% 0%
>> 65 cpu65                                                2%          2% 0%
>> 66 cpu66                                                2%          2% 0%
>> 67 cpu67                                                1%          1% 0%
>> 68 cpu68                                                0%          0% 0%
>> 69 cpu69                                                0%          0% 0%
>> 70 cpu70                                                6%          6% 0%
>> 71 cpu71                                                0%          0% 0%
>> 
>> 
>> [####@Borde - BGP] > inter monitor-traffic sfp6-LAN
>>                       name:   sfp6-LAN
>>      rx-packets-per-second:    215 812
>>         rx-bits-per-second:  908.3Mbps
>>   fp-rx-packets-per-second:    215 812
>>      fp-rx-bits-per-second:  908.3Mbps
>>        rx-drops-per-second:          0
>>       rx-errors-per-second:          0
>>      tx-packets-per-second:    264 707
>>         tx-bits-per-second:    2.0Gbps
>>   fp-tx-packets-per-second:    264 707
>>      fp-tx-bits-per-second:    2.0Gbps
>>        tx-drops-per-second:          0
>>       tx-errors-per-second:          0
>> -- [Q quit|D dump|C-z pause]
>> 
>> Probablemente puede ser un problema con algunos modelos, en nuestro caso
>> tenemos buenos resultados.
>> 
>> ??
>>>>> Buenas,
>>>>> comparto una mala experiencia que tuvimos con el primer despliegue de un
>>>>> CCR1072 como edge-router. La intención es evaluar con otros operadores
>>>>> si
>>>>> es reversible, así como también que sirva de antecedente para otros
>>>>> tentados por los 72 cores o la cantidad de SFP+.
>>>>> 
>>>>> Como RouterOS es un all-in-one es muy difícil comparar experiencias de
>>>>> otros dado que la combinatoria de features / protocolos que pueden estar
>>>>> activos es enorme. Por esto aclaro primero que nada el entorno de uso
>>>>> del
>>>>> equipo (y me atrevo a decir que seria el recomendado para todo router -y
>>>>> solo router- de frontera):
>>>>> 
>>>>> - Router de "borde" de un ISP.
>>>>> - IPv4-only (ni siquiera el package ipv6 enabled).
>>>>> - eBGP hacia dos carriers, aceptando e instalando solo default-gw.
>>>>> - iBGP a un segundo borde para redundancia active:active.
>>>>> - routing estático hacia adentro (no IGP, se podría decir que el único
>>>>> protocolo de routing activo es BGP).
>>>>> - Firewalling: sin estado (ninguna regla tiene connection-state como
>>>>> matcheo), sin reglas de Mangle, sin reglas de NAT,
>>>>> "connection-tracking=no".
>>>>> - No se brinda ningún servicio a la red (DNS, DHCP, NAT, etc) desde el
>>>>> router.
>>>>> - QoS: nada de nada, vacío de config el apartado queue.
>>>>> - Desactivado temporal por típica "a ver si es esto" en RouterOS: SNMP,
>>>>> todos los ip services menos winbox y ssh, logging.
>>>>> 
>>>>> 
>>>>> ## Maniobras realizadas a pedido del soporte Mikrotik que no
>>>>> tuvieron éxito (a pesar de que algunas rompen la necesidad o la idea de
>>>>> un
>>>>> equipo en producción original):
>>>>> - pasar por todas las last-version de cada rama de RouterOS (bugfix,
>>>>> current, RC).
>>>>> - disable de todas la reglas de firewall (que no hacían uso de
>>>>> connection-tracking).
>>>>> - usar varias interfaces SFP+ físicas individuales en vez de trunkear
>>>>> todo en vlans en unas pocas (ej 10G-WANs , 10G-CORE).
>>>>> - actualizar el firmware del RouterBoard.
>>>>> 
>>>>> ## Detalle *MUY IMPORTANTE:*
>>>>> - El CCR1072 se desplegó como remplazo natural del CCR1036 que estaba (y
>>>>> dado esto, sigue) en producción, pensando mayormente en eliminar los
>>>>> LAGs
>>>>> Nx1G y simplificar la conexión física a los carriers que se supera el
>>>>> Gbps
>>>>> de trafico contratado.
>>>>> - Por esta razón, en *cuanto a config el CCR1072 es un export/import
>>>>> literal de la config del CCR1036. Si hacemos un diff entre uno y otro
>>>>> solo varían las lineas referidas a las interfaces físicas* (incluso
>>>>> tratamos de imitar el uso de un LAG innecesario en el CCR1072 apostando
>>>>> a
>>>>> un tema de interrupciones).
>>>>> - Con esto quiero remarcar que a exactamente igual config y mismo
>>>>> tráfico
>>>>> agregado (equipo de remplazo) que el CCR1036, *el uso de CPU es horrible
>>>>> en el CCR1072*. Mal distribuido principalmente (haciendo uso de no mas
>>>>> de 4 o 5 cores) y generando peaks en cores individuales, combo que
>>>>> reduce
>>>>> enormemente la escalabilidad del equipo.
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> ?
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> ### Analizando con profile tool, en muchas ocasiones *"unclassified"*
>>>>> proccess es el que se lleva gran parte de los cores que se pican
>>>>> intercaladamente a valores altos con respecto a la carga de tráfico:
>>>>> 
>>>>> 
>>>>> CCR1072-BORDE] > tool profile cpu=2
>>>>> NAME                    CPU        USAGE
>>>>> firewall                  2           0%
>>>>> networking                2           1%
>>>>> management                2           0%
>>>>> routing                   2           0%
>>>>> idle                      2          82%
>>>>> profiling                 2           1%
>>>>> *unclassified              2          16% **<<< WTF!*
>>>>> 
>>>>> 
>>>>> CCR1072-BORDE] > tool profile cpu=2
>>>>> NAME                    CPU        USAGE
>>>>> firewall                  2           1%
>>>>> networking                2           0%
>>>>> management                2           0%
>>>>> routing                   2           0%
>>>>> idle                      2          65%
>>>>> profiling                 2           0%
>>>>> *unclassified              2          34% <<< WTF!*
>>>>> 
>>>>> 
>>>>> ### Incluso en horarios de poco trafico:
>>>>> 
>>>>> CCR1072-BORDE] > interface monitor-traffic sfp-sfpplus5
>>>>>                       name:  sfp-sfpplus5
>>>>>      rx-packets-per-second:        40 490
>>>>>         rx-bits-per-second:     387.9Mbps
>>>>>   fp-rx-packets-per-second:        40 490
>>>>>      *fp-rx-bits-per-second:     387.9Mbps*
>>>>>        rx-drops-per-second:             0
>>>>>       rx-errors-per-second:             0
>>>>>      tx-packets-per-second:        40 259
>>>>>         tx-bits-per-second:     387.6Mbps
>>>>>   fp-tx-packets-per-second:        40 259
>>>>>      *fp-tx-bits-per-second:     387.6Mbps*
>>>>>        tx-drops-per-second:             0
>>>>>       tx-errors-per-second:             0
>>>>> 
>>>>> ### Como se puede ver 60 o mas cores duermen la siesta mientras un par
>>>>> hacen todo el trabajo, en los 1072 reina el sindicalismo :P
>>>>> 
>>>>> 
>>>>> CCR1072-BORDE] > system resource cpu print
>>>>> # CPU
>>>>> 
>>>>> LOAD
>>>>>       IRQ        DISK
>>>>> 0 cpu0
>>>>> 
>>>>> 0%
>>>>>        0%          0%
>>>>> 1 cpu1
>>>>> 
>>>>> 0%
>>>>>        0%          0%
>>>>> 2 cpu2
>>>>> 
>>>>> 0%
>>>>>        0%          0%
>>>>> 3 cpu3
>>>>> 
>>>>> 0%
>>>>>        0%          0%
>>>>> 4 cpu4
>>>>>                                                          *
>>>>> 0%          0%          0%*
>>>>> * 5 cpu5
>>>>> 
>>>>> 37%          1%          0%*
>>>>> 6 cpu6
>>>>> 
>>>>> 0%
>>>>>        0%          0%
>>>>> 7 cpu7
>>>>> 
>>>>> 0%
>>>>>        0%          0%
>>>>> 8 cpu8
>>>>> 
>>>>> 0%
>>>>>        0%          0%
>>>>> 9 cpu9
>>>>> 
>>>>> 0%
>>>>>        0%          0%
>>>>> 10 cpu10
>>>>> 
>>>>> 0%
>>>>>        0%          0%
>>>>> 11 cpu11
>>>>> 
>>>>> 0%
>>>>>        0%          0%
>>>>> 12 cpu12
>>>>> 
>>>>> 0%
>>>>>        0%          0%
>>>>> 13 cpu13
>>>>> 
>>>>> 0%
>>>>>        0%          0%
>>>>> 14 cpu14
>>>>> 
>>>>> 0%
>>>>>        0%          0%
>>>>> 15 cpu15
>>>>> 
>>>>> 0%
>>>>>        0%          0%
>>>>> 16 cpu16
>>>>> 
>>>>> 0%
>>>>>        0%          0%
>>>>> 17 cpu17
>>>>> 
>>>>> 6%
>>>>>        6%          0%
>>>>> 18 cpu18
>>>>> 
>>>>> 0%
>>>>>        0%          0%
>>>>> 19 cpu19
>>>>> 
>>>>> 0%
>>>>>        0%          0%
>>>>> 20 cpu20
>>>>> 
>>>>> 0%
>>>>>        0%          0%
>>>>> 21 cpu21
>>>>> 
>>>>> 0%
>>>>>        0%          0%
>>>>> 22 cpu22
>>>>> 
>>>>> 0%
>>>>>        0%          0%
>>>>> 23 cpu23
>>>>> 
>>>>> 0%
>>>>>        0%          0%
>>>>> 24 cpu24
>>>>> 
>>>>> 0%
>>>>>        0%          0%
>>>>> 25 cpu25
>>>>> 
>>>>> 0%
>>>>>        0%          0%
>>>>> 26 cpu26
>>>>> 
>>>>> 0%
>>>>>        0%          0%
>>>>> 27 cpu27
>>>>> 
>>>>> 0%
>>>>>        0%          0%
>>>>> 28 cpu28
>>>>> 
>>>>> 0%
>>>>>        0%          0%
>>>>> 29 cpu29
>>>>> 
>>>>> 0%
>>>>>        0%          0%
>>>>> 30 cpu30
>>>>> 
>>>>> 0%
>>>>>        0%          0%
>>>>> 31 cpu31
>>>>> 
>>>>> 0%
>>>>>        0%          0%
>>>>> 32 cpu32
>>>>> 
>>>>> 0%
>>>>>        0%          0%
>>>>> 33 cpu33
>>>>> 
>>>>> 0%
>>>>>        0%          0%
>>>>> 34 cpu34
>>>>> 
>>>>> 0%
>>>>>        0%          0%
>>>>> 35 cpu35
>>>>> 
>>>>> 2%
>>>>>        2%          0%
>>>>> 36 cpu36
>>>>> 
>>>>> 0%
>>>>>        0%          0%
>>>>> *37 cpu37
>>>>> 
>>>>> 34%
>>>>>       33%          0%*
>>>>> 38 cpu38
>>>>> 
>>>>> 0%
>>>>>        0%          0%
>>>>> 39 cpu39
>>>>> 
>>>>> 2%
>>>>>        2%          0%
>>>>> 40 cpu40
>>>>> 
>>>>> 0%
>>>>>        0%          0%
>>>>> 41 cpu41
>>>>> 
>>>>> 0%
>>>>>        0%          0%
>>>>> 42 cpu42
>>>>> 
>>>>> 0%
>>>>>        0%          0%
>>>>> 43 cpu43
>>>>> 
>>>>> 0%
>>>>>        0%          0%
>>>>> 44 cpu44
>>>>> 
>>>>> 0%
>>>>>        0%          0%
>>>>> 45 cpu45
>>>>> 
>>>>> 0%
>>>>>        0%          0%
>>>>> 46 cpu46
>>>>> 
>>>>> 0%
>>>>>        0%          0%
>>>>> 47 cpu47
>>>>> 
>>>>> 1%
>>>>>        1%          0%
>>>>> 48 cpu48
>>>>> 
>>>>> 0%
>>>>>        0%          0%
>>>>> 49 cpu49
>>>>> 
>>>>> 0%
>>>>>        0%          0%
>>>>> 50 cpu50
>>>>> 
>>>>> 0%
>>>>>        0%          0%
>>>>> 51 cpu51
>>>>> 
>>>>> 0%
>>>>>        0%          0%
>>>>> 52 cpu52
>>>>> 
>>>>> 0%
>>>>>        0%          0%
>>>>> 53 cpu53
>>>>> 
>>>>> 0%
>>>>>        0%          0%
>>>>> 54 cpu54
>>>>> 
>>>>> 0%
>>>>>        0%          0%
>>>>> 55 cpu55
>>>>> 
>>>>> 0%
>>>>>        0%          0%
>>>>> 56 cpu56
>>>>> 
>>>>> 0%
>>>>>        0%          0%
>>>>> 57 cpu57
>>>>> 
>>>>> 0%
>>>>>        0%          0%
>>>>> 58 cpu58
>>>>> 
>>>>> 0%
>>>>>        0%          0%
>>>>> 59 cpu59
>>>>> 
>>>>> 0%
>>>>>        0%          0%
>>>>> 60 cpu60
>>>>> 
>>>>> 0%
>>>>>        0%          0%
>>>>> 61 cpu61
>>>>> 
>>>>> 0%
>>>>>        0%          0%
>>>>> 62 cpu62
>>>>> 
>>>>> 0%
>>>>>        0%          0%
>>>>> 63 cpu63
>>>>> 
>>>>> 0%
>>>>>        0%          0%
>>>>> 64 cpu64
>>>>> 
>>>>> 3%
>>>>>        2%          0%
>>>>> 65 cpu65
>>>>> 
>>>>> 0%
>>>>>        0%          0%
>>>>> 66 cpu66
>>>>> 
>>>>> 1%
>>>>>        0%          0%
>>>>> 67 cpu67
>>>>> 
>>>>> 0%
>>>>>        0%          0%
>>>>> 68 cpu68
>>>>> 
>>>>> 0%
>>>>>        0%          0%
>>>>> 69 cpu69
>>>>> 
>>>>> 0%
>>>>>        0%          0%
>>>>> 70 cpu70
>>>>> 
>>>>> 0%
>>>>>        0%          0%
>>>>> 71 cpu71
>>>>> 
>>>>> 0%
>>>>>        0%          0%
>>>>> 
>>>>> 
>>>>> --
>>>>> 
>>>>> *Ivan ChaperoÁrea Técnica y Soporte*
>>>>> Fijo: 03464-470280 (interno 535) | Móvil:  03464-155-20282  | Skype ID:
>>>>> ivanchapero
>>>>> --
>>>>> GoDATA Banda Ancha - CABLETEL S.A. | Av. 9 de Julio 1163 - 2183 -
>>>>> Arequito - Santa Fe - Argentina
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> _______________________________________________
>>>>> Lista mailing list
>>>>> Lista en arnog.com.ar
>>>>> http://mailmancabase.interdotnet.com.ar/mailman/listinfo/lista
>>>>> 
>>>>> 
>>>>> 
>>>> _______________________________________________
>>>> Lista mailing list
>>>> Lista en arnog.com.ar
>>>> http://mailmancabase.interdotnet.com.ar/mailman/listinfo/lista
>>>> 
>>>> 
>>>> _______________________________________________
>>>> Lista mailing list
>>>> Lista en arnog.com.ar
>>>> http://mailmancabase.interdotnet.com.ar/mailman/listinfo/lista
>>>> 
>>>> 
>>>> ------------ próxima parte ------------
>>> Se ha borrado un adjunto en formato HTML...
>>> URL: <http://mailmancabase.interdotnet.com.ar/pipermail/lista/
>>> attachments/20161127/86499f69/attachment.html>
>>> 
>>> ------------------------------
>>> 
>>> _______________________________________________
>>> Lista mailing list
>>> Lista en arnog.com.ar
>>> http://mailmancabase.interdotnet.com.ar/mailman/listinfo/lista
>>> 
>>> 
>>> Fin de Resumen de Lista, Vol 114, Envío 8
>>> *****************************************
>>> 
>> 
>> _______________________________________________
>> Lista mailing list
>> Lista en arnog.com.ar
>> http://mailmancabase.interdotnet.com.ar/mailman/listinfo/lista
>> 
>> 
>> 
>> 
>> -----
>> Se certificó que el correo no contiene virus.
>> Comprobada por AVG - www.avg.com
>> Versión: 2016.0.7924 / Base de datos de virus: 4728/13500 - Fecha de la
>> versión: 29/11/2016
>> 
>> 
>> 
>> _______________________________________________
>> Lista mailing list
>> Lista en arnog.com.ar
>> http://mailmancabase.interdotnet.com.ar/mailman/listinfo/lista
>> 
> 
> 
> 
> -- 
> 
> *Ivan ChaperoÁrea Técnica y Soporte*
> Fijo: 03464-470280 (interno 535) | Móvil:  03464-155-20282  | Skype ID:
> ivanchapero
> --
> GoDATA Banda Ancha - CABLETEL S.A. | Av. 9 de Julio 1163 - 2183 - Arequito
> - Santa Fe - Argentina
> ------------ próxima parte ------------
> Se ha borrado un adjunto en formato HTML...
> URL: <http://mailmancabase.interdotnet.com.ar/pipermail/lista/attachments/20161129/2b99e494/attachment.html>
> 
> ------------------------------
> 
> _______________________________________________
> Lista mailing list
> Lista en arnog.com.ar
> http://mailmancabase.interdotnet.com.ar/mailman/listinfo/lista
> 
> 
> Fin de Resumen de Lista, Vol 115, Envío 2
> *****************************************

------------ próxima parte ------------
Se ha borrado un adjunto en formato HTML...
URL: <http://mailmancabase.interdotnet.com.ar/pipermail/lista/attachments/20161129/26d28b77/attachment-0001.html>
------------ próxima parte ------------
A non-text attachment was scrubbed...
Name: Captura de pantalla 2016-11-29 a las 08.57.05.png
Type: image/png
Size: 119554 bytes
Desc: no disponible
URL: <http://mailmancabase.interdotnet.com.ar/pipermail/lista/attachments/20161129/26d28b77/attachment-0001.png>


Más información sobre la lista de distribución Lista