text:
Difference between revisions of "Scale-out Block Setup (ceph)"
m (→Create the OSDs and Journal Devices using Multi-Create)
m (→Setting up Scale-out SAN with Ceph)
|Line 13:
|Line 13:
= Setting up Scale-out SAN with Ceph =
= Setting up Scale-out SAN with Ceph =
|Line 122:
|Line 123:
= Management and Operation =
= Management and Operation =
Revision as of 15:13, 17 March 2016
Welcome to the Administrator's Guide to using QuantaStor's Scale-out SAN solution using Ceph. QuantaStor Scale-out SAN is intended to provide a highly scalable Storage Area Network solution where the QuantaStor appliance is used to provision SAN block storage to iSCSI or Fiber-Channel clients sourced from a Ceph storage backend.
This guide will provide details regarding Ceph technology as well as instructions on how to setup and configure a Scale-out SAN installation using QuantaStor.
- 1 Introduction to Scale-out SAN (iSCSI/FC) Block Storage using Ceph
- 2 Setting up Scale-out SAN with Ceph
- 2.1 Requirements Before Getting Started
- 2.2 System Installation and Initialization
- 2.3 Configure Front-end and Back-end Network Ports
- 2.4 Create QuantaStor Grid
- 2.5 Create Storage Targets / Hardware RAID Configuration
- 2.6 Network Time Protocol (NTP) Configuration
- 2.7 Create the Ceph Cluster
- 2.8 Note Before Creating OSDs and Ceph Journals
- 2.9 Create the OSDs and Journal Devices using Multi-Create
- 2.10 Create the Scale-out Storage Pools (Ceph Pools)
- 2.11 Provisioning Scale-out Storage Volumes (Ceph RBDs)
- 3 Management and Operation
- 3.1 Capacity Planning
- 3.2 Understanding the Cluster Health Dashboard
- 3.3 Adding a Node to a Ceph Cluster
- 3.4 Adding OSDs to a Ceph Cluster
- 3.5 Removing OSDs from a Cluster
- 3.6 Adding/Removing Monitors in a Cluster
Introduction to Scale-out SAN (iSCSI/FC) Block Storage using Ceph
Ceph is a highly-scalable storage platform designed to provide a single, distributed computer cluster to store data and provide interfaces for object, block, and file storage. Ceph is designed to reduce administration overhead and become fault tolerant through data replication between cluster nodes. As a result of its underlying design integrating distributed data replication, Ceph is both self-healing and self-managing.
This section will introduce the various component terminology and concepts regarding Ceph to enable confident creation and administration of a Scale-out SAN solution using QuantaStor and Ceph.
Ceph Cluster
A ceph cluster is a group of three or more systems that have been clustered together using the ceph storage technology. Ceph requires a minimum of three nodes to create a cluster which in turn establishes a quorum, Wikipedia Quorum (distributed computing).
In QuantaStor, systems must first be Grid members before they can be added to, or create, a Ceph cluster. In the above diagram, the QuantaStor Grid is also the Ceph Cluster. Note that when the Ceph Cluster is initially created there is no storage associated with it (OSDs), only monitors.
Ceph Monitor
The Ceph Monitors form a Paxos The Part-Time Parliament cluster for the management of cluster membership, configuration information, and state. Paxos is an algorithm (developed by Leslie Lamport in the late 80s) which uses a three-phase consensus protocol to ensure that cluster updates can be done in a fault-tolerant timely fashion even in the event of a node outage or node that is acting improperly. Ceph uses the algorithm so that the membership, configuration and state information is updated safely across the cluster in an efficient manner. Since the algorithm requires a quorum of nodes to agree on any given change an odd number of systems (three or more) are required for any given Ceph cluster deployment.
During initial Ceph cluster creation, QuantaStor will configure the first three systems to have active Ceph Monitor services. Configurations with more than 16 nodes should add at least two additional monitors. On configurations with more than 16 nodes two additional monitors should be setup. This can be done through the QuantaStor WebUI in the Scale-out Storage Configuration section.
Monitors startup automatically when the system starts. The status and health of monitors is monitored by QuantaStor, then displayed in the WebUI. A minimum of two ceph monitors must be online at all times. As an example, in a three node configuration two of the three systems must be online for the storage to be online and available.
When a cluster is initially created, QuantaStor configures the first three systems to have active Ceph Monitor services running.
Ceph Object Storage Daemon / OSD
The Ceph Object Storage Daemon, known as the OSD, is a daemon process that reads and writes data, representing the actual data storage containers. When a client writes data to a Ceph based iSCSI/RBD block device, or via the S3 gateway, the data is spread out across the OSDs in the cluster automatically.
QuantaStor Scale-out SAN with Ceph deployments must have at least 3x OSDs per system, making 9x OSDs total the minimum number of Daemons. Each OSD is attached to one BlueStore-based QuantaStor Storage Pool. QuantaStor requires the use of BlueStore Storage Pools for use as Ceph OSDs due to extended attribute requirements. Each OSD is also assigned one Journal Device.
Because the creation of OSDs, the underlying Storage Pools for them, and their associated Journal devices is a multi-step process QuantaStor has a Multi-Create configuration dialog which does all of these configuration steps for an entire cluster in a single dialog. This makes it easy to setup even hyper-scale Ceph deployments in minutes.
For additional BlueStore information see, New in Luminous: BlueStore.
Ceph Journals and Journal Devices
It is important to note that part of Ceph's design is to never cache writes. This is good and important because it ensures that every write is written to stable media (disk or SSD media) before Ceph acknowledges to a client that the write is complete. This applies to all writes irrespective of whether file, block, or object storage is configured. This design feature prevents corruption in the event of a power outage because the write transaction is only complete once the data is on stable media with redundancy. In the event of a system failure the cluster will automatically work around the bad node (essentially its collection of OSDs) until it comes back online and re-synchronizes with the cluster.
The trade-off to never caching writes is a loss of write performance, especially with spinning media. Hard-drives are slow due to rotational latency and seek times for spinning disk are high. The solution is log writes to very fast persistent solid state media (SSD, NVMe, XPoint, NVDIMM, etc). This write log is called a journal device and sometime a WAL device. (Technically in the Ceph architecture these are separate things but in practice the same media is used for both.)
Using a fast journal device allows Ceph to initially write data to the journal, returning a "write complete" to the client much much faster. Even though the data has not yet been written to the slower HDDs at that stage the data is on stable media so in the event of a power outage the log is automatically used to recover the in-flight writes. Ceph retains a copy of the data in RAM and uses that to write lazily to the HDD. This means that the journal device is only used as a write log and will never be read from unless a recovery scenario is encountered.
Because the journal device will encounter high, sustained write-pressure, Datacenter grade or Enterprise grade SSDs must be used for Ceph journal devices. NVMe and Optane based flash storage makes for the best journal devices. (Note: Desktop grade SSD devices generally do not have the necessary sustained write performance nor the endurance required to be used as a log device so they're unsuitable. As such OSNEXUS will not certify the use of any desktop media in any production deployment of any kind.)
Avoid Hardware RAID
As of QuantaStor 5 we no longer recommend the use of hardware RAID with Ceph configurations. It made sense to use it up until the Ceph Jewel release where the underlying storage used the FileStore layout (XFS based) because the hardware RAID helped improve journal performance. As of QuantaStor 5, the BlueStore layout is used exclusively so the use of hardware RAID is no longer needed or recommended.
Placement Group / PG
Ceph uses Placement Groups, PGs, to implement mirroring (or erasure coding) of data across OSDs according to the configured replica count for a given Ceph Pool.
The user specifies how many copies of the data must be maintained by the Ceph Pool during creation to ensure a level of high-availability and fault-tolerance, usually 2 copies when using hardware RAID or 3 copies when no disk-level RAID is present. Ceph in turn creates a series of Placement Groups as directed by QuantaStor to be associated with the Ceph Pool.
One way to think of the placement groups is as logical mini-mirrors in a RAID10 configuration. Each placement group is either a two-way, three-way or 4-way mirror across 2, 3, or 4 OSDs respectively. Because the number of OSDs will grow over the life of the cluster, QuantaStor allocates a large number of PGs for each Ceph Pool to evenly distribute data across the OSDs and accommodate future expansion as OSDs are added. In this way Ceph can very efficiently re-organize and re-balance PGs to mirror across new OSDs as they are added.
The PG count stays fixed as OSDs are added but a maintenance command can be run to increase the PG count for a Ceph Pool if the PG count gets low relative to the number of OSDs in the Pool. In general the PG count should be roughly 10x to 100x higher than the OSD count for a given Ceph Pool.
Similar to RAID10 technology, a PG can become degraded if one or more copies is offline. Ceph is designed to keep running in a degraded state when copies are lost, so whole systems can go offline without any disruption to clients accessing the cluster. Ceph also automatically repairs and updates the offline PGs once the offline OSDs come back online online and if the offline system doesn't come back online in a reasonable amount of time the cluster will auto heal itself by adjusting the PGs, swapping out the offline OSDs with good online OSDs. In this way a cluster will automatically heal a Ceph Pool back to 100% automatically (ie, return to full/complete copy count).
Also, if an OSD is explicitly removed, the PGs referencing it are re-balanced and re-organized across the remaining OSDs to recover the system back to 100% health on the remaining OSDs.
Object Storage Zone
S3 object storage gateways require the creation and management of several Ceph Pools, which together represent a region+zone for the storage of objects and buckets. QuantaStor groups all the Ceph Pools used to manage a given object storage configuration into a Object Storage Group or Zone. QuantaStor also automatically deploys and manages Ceph S3 Object Gateways on all systems in the cluster that were selected as gateway nodes when the Object Storage Group was created. Additional gateways can be deployed on new or existing nodes at any time via the web UI, CLI or REST API.
For additional information see Wikipedia, Ceph (software)
User Object Access Entries
Access to object storage via S3 requires a Access Key and a Secret Key just as with Amazon S3 storage. Each User Object Access Entry is an Access Key + Secret Key pair which is associated with a Ceph Cluster and Object Storage Group. You must allocate at least one User Object Access Entry to read/write buckets and objects to an Object Storage Group via the Ceph S3 Gateway.
Ceph CRUSH Maps and Resource Domains
Ceph supports the ability to organize placement groups, which provide data mirroring across OSDs, so that high-availability and fault-tolerance can be maintained even in the event of a rack or site outage. By defining failure-domains, such as a Rack of systems, a Site, or Building, a map can be created so that Placement Groups are intelligently laid out to ensure high-availability despite the outage of one or more failure-domains, depending on the level of redundancy.
This intelligent map is called the Ceph CRUSH map, standing for Controlled, Scalable, Decentralized Placement of Replicated Data, and it defines how to mirror data in the Ceph cluster to ensure optimal performance and availability.
Creating CRUSH maps manually can be a complex process, so QuantaStor creates and configures CRUSH maps automatically, saving a large degree of administrative overhead. To facilitate automatic CRUSH map management, detail regarding where each QuantaStor system is deployed must be provided. This is done by creating a tree of Resource Domains via the WebUI (or via CLI/REST APIs) to organize the systems in a given QuantaStor Grid into Racks, Sites, and Buildings. QuantaStor uses this information to automatically generate an optimal CRUSH map when pools are provisioned, ensuring optimal performance and high-availability.
Custom CRUSH map changes can still be made to adjust the map after the pool(s) are created and OSNEXUS provides consulting services to meet special requirements. Resource Domains are a QuantaStor construct so you will not find mention of them in general Ceph documentation, but they map closely to the CRUSH bucket hierarchy.
For additional information see, CRUSH MAPS
Setting up Scale-out SAN with Ceph
This section will deal step through the process of setting up and deploying Scale-out SAN using Ceph on QuantaStor. Before getting starting with setting up and deploying Scale-out SAN using Ceph, it is highly recommended that you familiarize yourself with the concepts in the above sections.
Requirements Before Getting Started
To achieve quorum a minimum of three appliances are required. The storage in the appliance can be SAS or SATA HDD or SSDs but a minimum of 1x SSD is required for use as a journal (write log) device in each appliance. Appliances must use a hardware RAID controller for QuantaStor boot/system devices and we recommend using a hardware RAID controller for the storage pools as well.
- 3x QuantaStor Storage Appliances minimum (up to 64x Appliances)
- Per Appliance:
- Intel Xeon or AMD Opteron CPU
- 1x 200GB or larger high write endurance SSD/NVMe/NVRAM device for use as Journal Device
- 4x Object Storage Daemons per Journal Device, maximum of 8x OSDs per Journal Device
- 5x to 100x HDDs or SSD for data storage per appliance
- 1x hardware RAID controller for OSDs (SAS HBA can also be used but RAID is faster)
- NTP / Time Synchronization
- Separate Networks for iSCSI and Ceph communication
Hardware RAID Configuration
Ceph has the ability to repair itself in the event of the loss of one or more OSDs which can be provisioned one-to-one with each Storage Pool which is one-to-one with a HDD. In QuantaStor deployments we support this style of deployment but our reference configurations always use hardware RAID to combine disks in to 5x disk RAID5 groups for several reasons:
- Disk failures have no impact on network load since they're repaired using a hot-spare device associated with the RAID controller
- Journals can be made fault-tolerant and easily maintained by configuring them into RAID1 or RAID5 units.
- Multiple DC grade SSDs can be combined to make ultra high-performance and high-endurance Journal Devices and the sequential nature of the journal writes lend well to the write patterns for RAID5.
- Disk drives are easy to replace with no knowledge of Ceph required. Simply remove the bad drive identified by the RED LED and replace it with a good drive. The RAID controller will absorb the new drive and automatically start repairing the degraded array.
- When the storage is fault-tolerant one need only maintain two (2) copies of the data (instead of 3x) so the storage efficiency is 40% usable vs. the standard Ceph mode of operation which is only 33% usable. That's a 20% increase in usable capacity.
- RAID controllers bring with them 1GB of NVRAM write-back cache which greatly boosts the performance of OSDs and Journal devices. (Be sure that the card has the CacheVault/MaxCache supercapacitor which is required to protect the write cache).
- Reduces the OSD and placement group count by 5x or more which allows the cluster to scale that much bigger and with reduced complexity (a cluster with 100,000 PGs use less RAM per appliance and is easier for Ceph monitors to manage than 500,000 PGs).
Besides the nominal extra cost associated with a SATA/SAS RAID Controller vs a SATA/SAS HBA we see few benefits and many drawbacks to using HBAs with Ceph. Some in the Ceph community prefer to let bad OSD devices fail in-place never to be replaced as a maintenance strategy. Our preference is to replace bad HDDs to maintain 100% of the designed capacity for the life of the cluster. Hardware RAID makes that especially easy and QuantaStor has integrated management for all major RAID controller models (and HBAs) via the Web UI (and CLI/REST). QuantaStor's web UI also has an enclosure management view so that it's easy to identify which drive is bad and where it is located in the server chassis.
Front-end / Back-end Network Configuration
Networking for scale-out file and block storage deployments use a separate front-end and back-end network to separate the client communication to the front-end network ports (S3/SWIFT, iSCSI/RBD) from the inter-node Ceph communication on the back-end. This not only boosts performance, it increases the fault-tolerance, reliability and maintainability of the Ceph cluster.
All nodes should have one or more ports designated as front-end ports and assigned IP address and subnet masks specifically for client access. For all nodes one or more ports should be designated as the front-end ports and assigned appropriate IP addresses and subnets to enable client access. One can have multiple physical, virtual IPs, and VLANs used on the front-end network to enable a variety of clients to access the storage. The back-end network ports should all be physical ports but it is not required.
A basic configuration Ceph network configuration:
System Installation and Initialization
The minimum node count for a Scale-out SAN configuration using Ceph is 3 Systems, with a maximum node count of 64. Install QuantaStor on each System and following the steps below to complete the initial setup. For details, please see the QuantaStor Installation Guide, which includes details steps and troubleshooting.
- Login to the QuantaStor web management interface on each system
- Add your license keys, one unique key for each system
- Right-click on the storage system, choose 'Modify Storage System..' and set the DNS IP address (eg: 184.108.40.206), and the NTP server IP address (important!)
Configure Front-end and Back-end Network Ports
Update the configuration of each network port using the Modify Network Port dialog to put it on either the front-end network or the back-end network. The port names should be consistently configured across all appliance nodes such that all ethN ports are assigned IPs on the front-end or the back-end network but not a mix of both.
Create QuantaStor Grid
A QuantaStor Grid enables the administration and management of multiple Systems as a unit (single pane of glass). By joining Systems together in a Grid, the WebUI will display and allow access to resources and functionality on all Systems that are members of the Grid. Grid membership is also a prerequisite for the High-Availability and Scale-out configurations offered by QuantaStor.
Networking between the nodes must be configured before proceeding with Grid setup. Once network is configured and confirmed on a per-System basis, proceed to creating the QuantaStor grid using the following instructions.
Create the Grid on the First Node
The node where the Grid is created initially will be elected as the initial primary/master node for the grid. The primary node has an additional role in that it acts as a conduit for intercommunication of grid state updates across all nodes in the grid. This additional role has minimal CPU and memory impact.
- Select the Storage Management tab and click the Create Grid button under the Storage System Grid section, or right-click on the System under Storage System and select Create Management Grid
- Name: The Grid name can be set to anything.
After pressing OK, QuantaStor will reconfigure the node to create a single-node Grid.
Add Remaining Nodes to the Grid
Now that the Grid is created and the Primary node is a member, proceed to add all the additional systems. Note that this should be done from the Primary node's WebUI.
- Click on the Add System button in the Storage Management ribbon bar or right click on the Grid and select Add System to Grid...
- IP or hostname for the node to add
- Username for an Administrative user (default is admin)
- Password to authenticate
Repeat this process for each node to be added to the QuantaStor Grid. The Grid and member Systems can be managed by connecting to the WebUI of any of the members. It is not necessary to connect to the master node.
Preferred Grid IP
System to System communication typically works itself out automatically but it is recommended that you specify the network to be used for system inter-node communication for management operations. This is done by selecting the "Preferred Grid Port IP" from the in the "Network Settings" tab of the Storage System Modify dialog by right-clicking on each system in the grid and select 'Modify Storage System...'.
Note regarding User Access Security
Be aware that the management user accounts across the systems will be merged as part of joining the Grid. This includes the admin user account. In the event that there are duplicate user accounts, the user accounts in the currently elected primary/master node takes precedence.
Create Storage Targets / Hardware RAID Configuration
As noted in the Getting Started section, creating RAID5 pools using 5 disks each to use as your XFS Storage Pool devices is highly recommended for performance and redundancy reasons. Ultimately any form of block storage can be used to create the XFS Storage Pools necessary for use with Ceph, but any other configuration should be reserved only for test environments.
- Expand the Hardware Enclosures & Controllers drawer under the Storage Management window. Right click on the Controller and select Create Unit.
Proceed to create hardware RAID5 unites using 5 disks per RAID unit (4 data disks + 1 Parity). Repeat this on each node of the Grid until all devices have been assigned to RAID units and presented as LUNs. Click the Scan button in the Physical Disks section of the ribbon bar to discover the new devices.
Network Time Protocol (NTP) Configuration
NTP is a system to make sure that the clock on computers is accurate. It is particularly important in Ceph cluster deployments that the clock be accurate. When the clocks on two or more systems are not synchronized it is called clock skew. Clock skew can happen for a few different reasons, most commonly:
- NTP server setting is not configured on one or more systems (use the Modify Storage System dialog to configure)
- NTP server is not accessible due to firewall configure issues
- No secondary NTP server is specified so the outage of the primary is leading to skew issues
- NTP servers are offline
- Network configuration problem is blocking access to the NTP servers
Ensure NTP servers are configured for each System by right clicking on the System under the Storage System drawer and select Modify Storage System... and examine that you have valid NTP servers configured.
- Note that QuantaStor retains the Ubuntu default NTP servers, but this may need to be adjusted based on accessibility restrictions of the Ceph cluster's network.
Create the Ceph Cluster
The QuantaStor Grid and a Ceph Cluster are separate constructs. The Grid must be setup first and can consist of heterogeneous mix of up to 64 systems which can span sites. Within the grid one can create up to 20x Ceph Clusters where each cluster must have at least 3x systems. A given systems cannot be a member of more than one grid or cluster at the same time. Typically a grid will consist of just one or two Ceph clusters and most often the cluster is built using systems that are all within one site but a cluster can span multiple sites as long as the network connectivity is stable and the latency is relatively low. For high latency links the preferred method is to setup a asynchronous remote-replication link to transfer data (delta changes) from a primary to a secondary site based on a schedule.
- Navigating to Scale-out Block & Object Storage in the top menu, then click on Create Cluster in the ribbon bar
- Alternatively you can right-click in the empty space below the Ceph Cluster dialog and select Create Ceph Cluster
Make sure that all of the network ports selected for the front-end network are the ports that will be used for client access (S3, iSCSI/RBD) and that the back-end ports are on a private network.
Note Before Creating OSDs and Ceph Journals
The Object Storage Daemons (OSDs) store data, while the Journals accelerate write performance as all writes flow to the journal and are complete from the client perspective once the journal write is complete. A quick review of OSD and Journal requirements in a QuantaStor/Ceph Scale-out Block and Object configuration:
- Each system should have 5x HDDs (or SSDs) for data devices to satisfy minimum Placement Group redundancy requirements
- Each system should have 1x or more journal devices (NVMe/Optane preferred)
Create the OSDs and Journal Devices using Multi-Create
The Multi-Create tool makes the creation of the Ceph Journals and OSDs very simple, with a single dialog handling the initialization of all components. Use the following instructions to begin.
- Select the Scale-out Storage Configuration tab in the top menu of the QuantaStor WebUI and click the Multi-Create button from the Storage Media section of the ribbon bar.
A dialogue box titled Create Object Storage Daemons/Devices (OSDs) will open:
- Mark the devices intended for use as Journals from each system
- Click the >' arrow by the Journal (WAL) Devices section to set the devices as Journal devices
- If Journal devices already exist in the Ceph Cluster the Use available Ceph Journal partitions checkbox can be marked
- Next select the remaining disks to be used as OSDs and click the > arrow to add them to the OSD list
- Once satisfied with the arrangement, press OK to initiate Journal and OSD creation
Once this form is submitted, QuantaStor will begin the process by creating an BlueStore Storage Pool for each of the devices added as an OSD in the Multi-Create dialogue. This process takes time. Allow a few minutes after the final Create Ceph OSD task completes for all OSDs to show up and fully initiate.
Please see the Manual Journal and OSD creation section in Management and Operation below for details on how to manually build the Journals, BlueStore Storage Pools, and OSDs. Because of the number of steps required and complexity, manual creation discouraged in favor of the Multi-Create tool.
Create the Scale-out Storage Pools (Ceph Pools)
The Ceph Storage Pool represents the data container that RBD Scale-out Storage Volumes will be generated from. By default, the creation of a new Ceph Storage Pool will select all OSDs that are available in the Ceph Cluster. Once the Ceph Pool is created the Cluster health status will update to reflect the overall health of the cluster to include the health of the PGs associated with the new pool.
To create the Ceph Storage Pool begin by clinking Create button in the Ceph Storage Pools section of the ribbon bar:
The Create Ceph Storage Pool dialogue will pop-up:
- Name: A descriptive name for administrative purposes
- Ceph Cluster: Select the appropriate cluster if the Grid includes multiple Ceph Clusters
- Pool Layout: Use this to set your level of redundancy for the cluster.
- Options:
- Replica-2 - Ceph will maintain two copies of the data across the cluster
- Replica-3 - Ceph will maintain three copies of the data across the cluster
Note that in QuantaStor 4.0 and below, all OSDs in the cluster will be included in the pool, but the QuantaStor 4.1 release will include the ability to select specific OSDs to be used for data and other OSDs to be used as a fast SSD tier to accelerate read and write performance.
Provisioning Scale-out Storage Volumes (Ceph RBDs)
Provisioning iSCSI Storage Volumes from Ceph based Storage Pools is identical to the process used for ZFS based Storage Pools. You can use the Create Storage Volume dialog from the main Storage Management tab and select the pool to provision from. Note that the iSCSI initiator (VMware, Linux, Windows, etc) must be configured login to a QuantaStor network port (target port) IP address on each appliance associated with the Ceph Cluster that the RBD is a member of so that there are multiple paths to the storage. If only a single appliance connection is made there will be no automatic fail-over in the event of an outage.
To create Scale-out Storage Volumes (RBDs) begin by clinking Create button in the Scale-out Storage Volumes(RBD) section of the ribbon bar:
The Create Ceph Storage Volume dialogue will pop-up:
- Name: A descriptive name for administrative purposes
- Ceph Storage Pool: If multiple pools are available in the Grid, select the appropriate pool
- Ceph Storage Pool Information is displayed for informational purposes
- Size: Select the size of the Volume to be created. Note that all Ceph/RBDs are Thin provisioned
Management and Operation
All key setup and configuration options are completely configurable via the WebUI. Operations can also be automated using the QuantaStor REST API and CLI. Custom Ceph configuration settings can also be done at the console/SSH for special configurations, custom CRUSH map settings; for these scenarios we recommend checking with OSNEXUS support or pre-sales engineering for advice before making any major changes.
Capacity Planning
One of the great features of QuantaStor's scale-out Ceph based storage is that it is easy to expand by adding more storage to existing appliances or by adding more appliances into the cluster.
Expanding by adding more Appliances
Note that it is not required to use the same hardware and same drives when you expand but it is recommended that the hardware be a comparable configuration so that the re-balancing of data is done evenly. Expanding can be done one appliance at a time and the OSDs for the new appliance should be roughly the same size as the OSDs on the other appliances.
Expanding by adding storage to existing Appliances
If you add more OSDs to existing appliances then be sure to expand multiple or all appliances with the same number of new OSDs so that the re-balancing can work efficiently. If your pools are setup with a replica count of 2x then at minimum a pair of appliances with additional OSDs at a time.
Understanding the Cluster Health Dashboard
The cluster health dashboard has two bars, one to show how much space is used and available, the second shows the overall health of the cluster. The health bar represents the combined health of all the "placement groups" for all pools in the cluster.
If a node goes offline or a system is impacted such that the OSDs become unavailable this will cause the health bar to show some orange and/or red segments. Hover the mouse over the effected section to get more detail about the OSDs that are impacted.
Additional detail is also available if the OSD section has been selected. If you've setup the cluster with OSDs that are using hardware RAID then your cluster will have an extra level of resiliency as disk drive failures will be handled completely within the hardware RAID controller and will not impact the cluster state.
Adding a Node to a Ceph Cluster
Additional Appliances can be added to the Ceph Cluster at any time. The same hardware requirements apply and the Appliance will need to have appropriate networking (connections to both Client and Backend networks).
To add an additional Appliance to the Ceph Cluster:
- First add the Appliance to the QuantaStor Grid
- Select the Scale-out Block & Object Storage menu and click on Add Member under the Ceph Clusters section of the ribbon bar:
The Add Member to Ceph Cluster dialogue box will pop-up:
- Ceph Cluster: If there are multiple Ceph Clusters in the grid, select the appropriate cluster for the new member to join
- Storage System: Select the Appliance to attach to the Ceph Cluster
- Client & Backend Interface: QuantaStor will attempt to select these interfaces appropriately based on their IP addresses. Verify that the correct interfaces are assigned. If the interfaces do not appear, ensure that valid IP addresses have been assigned for the Client and Back-end Networks and that all physical cabling is connected correctly.
- Enable Object Store for this Ceph Cluster Member: Leave this unchecked if using as Scale-out SAN/Block Storage solution. Check this if using Object storage.
Adding OSDs to a Ceph Cluster
OSDs can be added to a Ceph Cluster at any time. Simply add more disk devices to existing nodes or add a new member to the Ceph Cluster which has unused storage. It is recommended to setup RAID5 hardware RAID units using the available spindles using the Hardware Controllers & Enclosures section. Afterwards the new devices can be added as OSDs using the Multi-OSD Create dialog.
Note that if you are not adding additional SSD devices to be used as journal devices for the new OSDs that you must check the option to Use existing journal devices which will use existing unused journal partitions for the newly create OSDs.
Removing OSDs from a Cluster
OSDs can be removed from the cluster at any time and the data stored on them will be rebuilt and re-balanced across the remaining OSDs. Key things to consider include:
- Make sure there is adequate space available in the remaining OSDs in the cluster. If you have 30x OSDs and you're removing 5x OSDs then the used capacity will increase by roughly 5/25 or 20%. If there isn't that much room available be sure to expand the cluster first, then retire/remove old OSDs.
- If there is a large amount of data in the OSDs it is best to re-weight the OSD gradually to 0 rather than abruptly removing it.
- In multi-site configurations especially, make sure that the removal of the OSD doesn't put pools into a state where there are not enough copies of the data to continue read/write access to the storage. Ideally OSDs should be removed after re-weighting and subsequent re-balancing has completed.
- Select the Scale-out Block & Object Storage menu and click on Delete under the Object Storage Daemon section of the ribbon bar:
This will open the Delete a Ceph Storage Daemon pop-up:
- Selecting the OSD will display information about it
- Deletion will take time, depending on how much data needs to be migrated to other OSDs in the Cluster
Adding/Removing Monitors in a Cluster
For Ceph Clusters up to 10x to 16x appliances the default 3x monitors is typically fine. The initial monitors are created automatically when the cluster is formed. Beyond the initial 3x monitors it may be good to jump up to 5x monitors for additional fault-tolerance depending on what the cluster failure domains look like. If you cluster is spanning racks, it is best to have a monitor in each rack rather than having all the monitors in the same rack which will cause the storage to be inaccessible in the event of a rack power outage. Adding/Removing Monitors is done by using the buttons of the same names in the toolbar.


text:
September 8, 2016 to September 8, 2017 – Worldwide Want to change your look with new abaya designs on this festival season? Mirraw giving you best deals on latest abaya designs on this festival season. Mirraw organizing "Kamaal Dhamaal Sale". We are… Organized by Sarvesh S. | Type: shopping
November 23, 2016 to November 23, 2017 – Online On this Cyber Monday and Black Friday at Mirraw we are giving special discounts to our loyal customers. We are giving upto 70% Off on your every purchase. You can shop women apparels, fashion accesso… Organized by Sarvesh S. | Type: shopping
December 3, 2016 to December 3, 2017 – 8239 93rd Avenue North Brooklyn Park, Minnesota 55445 If you want to begin juicing, it is useful to have helpful advice. The healthier juicing strategies you are about to understand will help you begin on the right foot. As you begin to use these sugges… Organized by Wholesale e Liquid | Type: wholesale, e, liquid
December 5, 2016 to December 5, 2017 – 8239 93rd Avenue North, Brooklyn Park, MN, 55455 Most wholesalers deliver daily or two after the order was placed based on your location. In case you are an e-Liquid wholesaler, distributor or reseller and are considering ordering wholesale JUICY e… Organized by wholesale e liquid | Type: wholesale, e, liquid
December 8, 2016 to November 30, 2017 – Mirraw.com Everyone loves online shopping for clothing and that never ends. That's why Mirraw brings you the after festival sale. This Fashion carnival is for all fashion lovers. Checkout the latest fashionable… Organized by Sarvesh S. | Type: online, shopping
December 19, 2016 to December 19, 2017 – Glenview Many entrepreneurs go into construction consulting and remodeling business for themselves to be in a position to make money doing something they enjoy, despite the risks that might entail. Thorough m… Organized by Chicago Custom Home Builder | Type: chicago, custom, home, builder
December 20, 2016 to December 20, 2017 – Baltimore Managing an emergency locksmith service business is far easier if you have a clear vision. There will be difficulties you will face when attempting to meet the goals you set. Implement these suggesti… Organized by Locksmith in Baltimore | Type: locksmith, in, baltimore
December 21, 2016 to December 21, 2017 – Beverly Hills, Then facelift is the very best choice to avoid and refresh your appearance, when you have started to get the hint of aging on your own face. Prior to going for this particular process if you're looki… Organized by Los Angeles Facelift | Type: los, angeles, facelift
March 14, 2017 at 6pm to December 11, 2017 at 5pm – India Online term insurance is one of the simplest term life insurance plans in India providing a large coverage for a fixed period. Online term insurance is a cost-effective way to avail substantial death… Organized by Shree Joshi | Type: insurance, tips
March 15, 2017 to December 31, 2017 – Mumbai Our Educational Loan In India are designed to suit diverse needs of students who wish to pursue higher studies in India & abroad. We provide 100% education finance. Organized by Shyamoli Shah | Type: general
May 13, 2017 all day – usa UFC 211 UFC 211 UFC 211 Live UFC 211 Live Stream UFC 211 Fight Card UFC 211 TV Coverage UFC 211 Miocic vs Dos Santos 2 Miocic vs Dos Santos 2 https://211-ufc.org/ https://211-ufc.org/ https://211-ufc… Organized by livetvs | Type: ufc, 211

subset: commoncrawl


text:
updated 6/19/2011 12:29:51 PM ET
MR. DAVID GREGORY: This Sunday, Obama at war. In Libya, several months of airstrikes have failed to topple Moammar Khaddafy, and some House and Senate leaders are now demanding that the White House seek congressional approval for continued military action.
REP. JOHN BOEHNER (R-OH): I think it's time for the president to outline to the American people why we are there, what the mission is, and, and what our goals are, and how do we exit this?
(End videotape)
MR. GREGORY: In Afghanistan, the debate heats up over just how many American soldiers should return home this summer. And the cost of war is a huge factor as budget talks between Democrats and Republicans grind on. A debate this morning between assistant majority leader in the Senate, Democrat from Illinois Senator Dick Durbin, and Republican senator from South Carolina, member of the Armed Services Committee, Senator Lindsey Graham.
Then, the race for the White House, and the GOP field taking center stage in New Hampshire.
REP. MICHELE BACHMANN (R-MN): I want to announce tonight, President Obama is a one-term president.
FMR. GOV. MITT ROMNEY: This president has failed, and he's failed at a time when the American people counted on him to create jobs and get the economy going.
FMR. GOV. TIM PAWLENTY: This president is a declinist. He views America as one of equals around the world.
(End videotape)
MR. GREGORY: As the candidates keep the focus on the president, who will emerge as Obama's biggest challenge?
And the final act in the Weiner scandal.
FMR. REP. ANTHONY WEINER (D-NY): So today I'm announcing my resignation from Congress.
Offscreen Voice: Yeah!
(End videotape)
MR. GREGORY: How do Democrats now refocus their agenda? This morning, our political roundtable weighs in: the mayor of Los Angeles and president-elect of the U.S. Conference of Mayors, Antonio Villaraigosa; editorial page editor for The Wall Street Journal, Paul Gigot; presidential historian Doris Kearns Goodwin; chief White House correspondent and political director for NBC News, Chuck Todd; and NBC's chief foreign correspondent, Richard Engel.
Announcer: From NBC News in Washington, MEET THE PRESS with David Gregory.
MR. GREGORY: Good morning. A bipartisan golf game captured the eyes of Washington as the president of the United States and the speaker of the House, John Boehner, teamed up yesterday against Vice President Biden and Republican governor of Ohio, John Kasich. These leaders are still miles apart on critical issues: the budget, the future of the economy, and the president's foreign policy. We're going to get into those debates this morning with two key Senate leaders: the assistant Democratic leader, Dick Durbin; and Republican senator Lindsey Graham.
Welcome to both of you. I want to start on the budget and the future of the economy because these two sides, Democrats and Republicans, are still far apart.
And, Senator Durbin, the big question, if you know the contours of the debate here, are Republicans going to deal on taxes? Are Democrats going to really cut spending on issues like Medicare? Where does the breakthrough come from?
SEN. DICK DURBIN (D-IL): Well, I think Vice President Biden is leading this bipartisan effort, and progress is being made. That's a good thing. And I hope they can reach an agreement, at least on a short-term budget agreement, that leads to passing the debt ceiling ahead of time. We shouldn't wait till high noon to pass this debt ceiling because, I can tell you, it'll mean higher interest rates if we don't, it'll mean stalling our economy even further and risking another recession.
MR. GREGORY: But we know the stakes. Senator, we know the stakes. Where does the breakthrough come from specifically?
SEN. DURBIN: Well, I can tell you the--that the long-term breakthrough, I hope, engages everyone, both parties, to put everything on the table. Let me underline the word everything. That's painful for Democrats because we're talking about entitlement programs, painful for Republicans because we're speaking of revenue. But I think the president's deficit commission got it right. If we put it all on the table, we can do it in a sensible fashion.
MR. GREGORY: So--and would you then support a package that involved cuts to Medicare benefits if that's what was required for a grand bargain?
SEN. DURBIN: I wouldn't go--I certainly wouldn't go as far as the House Republican budget, which I think would eliminate Medicare as we know it today. But there are ways to make health savings in Medicare, health savings that will mean that it'll be a solvent program for so much longer. We shouldn't, though, break the basic promise to the American people that Medicare is going to protect them in their retirement.
MR. GREGORY: Senator Graham, I hear in that something that I've heard from my own reporting at the White House, which is that Democrats may be willing to deal on actual cuts to Medicare benefits--it could be means testing, it could be raising the retirement age. But that would have to, to create some room for Republicans to deal on revenues. Maybe not tax rates, but doing some things to create some extra revenues for the government. Do you see that as something you could support?
SEN. LINDSEY GRAHAM (R-SC): Well, what Dick and the gang of six was trying to do was exactly that, take the $1.2 trillion in deductions that we have that we give out to special interest groups, eliminate some of those deductions. That will increase revenue and pay off the debt. No one on the Republican side's going to vote to raise taxes. But I think many of us would look at flattening the tax code, do away with deductions and exemptions, and take that revenue to help pay off the debt; repatriate money overseas...
MR. GREGORY: So you're not opposed to some, some moves that could actually create some new revenue?
SEN. GRAHAM: Yeah, one way to do that is to do away with ethanol subsidy and a bunch of other subsidies that go to a few people, take that money back into the federal treasury and pay off the debt. That doesn't raise taxes, that takes special interest groups, their gravy train away and helps pay off the debt. That's one way to help pay off the debt.
MR. GREGORY: To both of you, and I'll start with Senator Durbin, look, what's going on around the world is something that we're debating here in America. Look at the images that came out of Greece this week as you've got governments that are defaulting in Europe potentially, big cuts in public spending. And this is the result, rioting in the streets. There are union members and there are just citizens who are saying this is way too much. Senator Durbin, are we headed in this direction with the kind of actions we're talking about in terms of cutting public spending? Could we have that kind of reaction here?
SEN. DURBIN: I don't think so. We can avoid that if we act like grown-ups. Let me go back to your point, David. I did not call for Medicare benefit cuts. We've got to keep our promise to seniors that Medicare will be there to protect them. But there are ways to make savings in Medicare, ways to put in some competition, for example, in the prescription drug program that will dramatically reduce the cost. But when it comes to America making a sacrifice so that we can move forward, I think Americans have always rolled up their sleeves and pitched in to do it. Keep in mind, for every dollar we spend in Washington, we borrow 40 cents, primarily from China, our major competitor in the world. This has to come to an end, and it can. We can have a sensible, responsible discussion based on the Bowles-Simpson commission if we put everything on the table and respect the basic programs, Social Security and Medicare, that people count on in their retirement.
MR. GREGORY: All right, but let me pin you down on Medicare, since you brought it up again. Could you see raising--supporting raising the retirement age or means testing benefits?
SEN. DURBIN: No. Let me tell you, I...
MR. GREGORY: Well, then where are the meaningful savings that you get out of--you say you don't want to break the promise. The reality is it's not sustainable for the future for the American people. Is that not true?
SEN. DURBIN: David, let me tell you, saying to people wait two more years for Medicare is not a good idea. Think about how vulnerable people are at that age. Maybe they're retired, at this point have no health insurance, Medicare is their lifeline to basic healthcare protection. And I think the House Republican budget went too far. We don't want to go that far.
MR. GREGORY: All right, I want to--Senator Graham, let me get back to this other point, which is what we're seeing going on in the eurozone right now, potential defaults. Is there a risk, from your side of the aisle, that these draconian cuts in spending that so many Americans think are necessary may actually halt what we're still seeming--seeing as a very fragile, very weak economic recovery?
SEN. GRAHAM: Well, the 2010 election was about what? A rejection of the Democratic Congress' explosion of spending. It's not accident that we picked up the House after the stimulus package, Obama health care. The American people saw what was going on with doubling and tripling the debt, and they said no. There's no way on God's green earth you're going to balance the budget until you put entitlements on the table, like Ronald Reagan and Tip O'Neill. David, what did they do? They adjusted the age slowly but surely from 65 to 67. We need to means test benefits. Everybody on this program could, in the future, give up some of their benefits for Social Security to keep it solvent. And you've got to do the same thing on Medicare, slowly but surely adjust the age, and upper income Americans should pay more when it comes to Medicare.
MR. GREGORY: But interesting that your...
SEN. GRAHAM: Seventy-five cents of every...
MR. GREGORY: ...your party's not with you on that. The party--I mean, Paul Ryan didn't even include Social Security in his budget plan, and these are still areas...
SEN. GRAHAM: Well, well, that...
MR. GREGORY: You say we have to act like grown-ups. Nobody's really willing to go in and touch these things.
SEN. GRAHAM: Ronald Reagan and Tip O'Neill were grown-ups. They came up with a formula to save Social Security that slowly but surely adjusted the age. People over 55 are unaffected, but I think young Americans believe Social Security and Medicare are going to fail if we don't do something. Why should you and I get 100 percent of our Medicare paid--75 cents of every dollar for our Medicare payments come out of general treasury. We should be paying that ourselves because we can afford to.
MR. GREGORY: All right, let me spend a few minutes on foreign policy here because Congress Republicans really bucking the president's version of foreign policy. And let me start with Libya, the fight over whether the president needs specific authority from Congress to wage this effort. And the big question, are there actually hostilities involved? The White House issued a report saying U.S. operations do not involve sustained fighting or active exchanges of fire with hostile forces, nor do they involve U.S. ground troops. Are there hostilities? House Speaker Boehner said, "Come on, this is ridiculous."
(Videotape, Thursday)
REP. BOEHNER: We're part of an effort to drop bombs on Khaddafy's compounds. I don't know--I just--it doesn't pass the straight face test, in my view, that we're not in the midst of hostilities.
(End videotape)
MR. GREGORY: So why shouldn't there be specific approval from Congress to wage whatever we're waging there, Senator? Senator Graham, I'll start with you.
SEN. GRAHAM: Oh, you talking to me, David?
MR. GREGORY: Yeah.
SEN. GRAHAM: Well, you know, one, I would take the course that conservatives have been taking for the last 30 years. The War Powers Act is unconstitutional, not worth the paper it's written on. It requires congressional approval before the commander in chief can commit troops after a certain period of time, and it would allow troops to be withdrawn based on the passage of a concurrent resolution never presented to the president. So I think it's an infringement on the, the, the power of the commander in chief. The president's done a lousy job of communicating and managing our involvement in Libya, but I will be no part of an effort to defund Libya or to try to cut off our efforts to bring Khaddafy down. If we fail against Khaddafy, that's the end of NATO. Egypt's going to be overrun and the "Mad Dog of the Mideast," what Ronald Reagan called Khaddafy, if he survives this, you're going to have double the price of oil that you have today because he will take the whole region and put it in, into chaos. And I will be--I won't be any part of that. So from my Republican point of view, the president needs to step up his game in Libya, but Congress should sort of shut up and not empower Khaddafy. Because he wrote a letter to the Congressional leadership basically thanking them for their involvement in trying to end this conflict.
MR. GREGORY: Well, it's interesting, Senator Durbin, you are on opposite sides of this. You do think the president should get specific authority from Congress to do this. What about the view that you're sort of giving some aid and comfort here to Republicans who simply just want to buck the president on any foreign policy goal. Your own leader in the Senate, Harry Reid, said, "No, we don't need a War Powers approval resolution here." Yet, you disagree.
SEN. DURBIN: I respect Harry Reid very much and certainly the president, as you know, but I respectfully disagree. It's true. The War Powers Act is an infringement on the president's power as commander in chief. So is the Constitution, which makes it clear the American people make decisions about going to war through members of Congress. The president's doing the right thing. What we have here, this would be "butcher of Benghazi," Khaddafy, needs to be stopped so he doesn't kill innocent people. The president brought together the Arab League, the United Nations, and NATO and said we are going to play a supportive rule--role, no ground troops. We're going to have a limited duration conflict to stop Khaddafy. That was the right thing. But I think that the War Powers Act and Constitution make it clear that hostilities by remote control are still hostilities. We are killing with drones what we would otherwise be killing with fighter planes. And we are engaged in hostilities in Libya. What we should do is act on a timely basis to pass congressional authorization under the War Powers Act. I reject the Republican approach, which has been suggested by Speaker Boehner and others to cut off the troops. It would give solace to Khaddafy. It would undermine the people who are resisting him in that nation, and I agree completely with Lindsey Graham. It would call into question the future of NATO.
MR. GREGORY: All right. But--well, first of all, we don't have any troops over there. Senator Graham, the reality is, what are we doing? What is the endgame? I mean, you talk about it would be the end of NATO, but what NATO has done for several months hasn't worked. So, I mean, there is this criticism of you...
SEN. GRAHAM: Well, I would argue...
MR. GREGORY: ...and others which is that the advocacy for sort of perpetual war when we don't seem to have a plan.
SEN. GRAHAM: Well, we do. We had an opportunity to end this very quickly. The day you took American air power out of--off the table, NATO became a weakened organization. But we are making progress. Khaddafy is on his last leg. The rebels are getting stronger. They've taken the fight to Tripoli. I said about four weeks ago, "Go after Khaddafy's inner circle, break their will." We're pounding Tripoli. But the big mistake was to take American air power off the table. What I would like to see is for America to rejoin NATO when it comes to an aerial bombardment. We don't need ground troops. And if you don't think Khaddafy surviving affects America's national security interests. We're just on different planets. If this guy survives, it's the end of NATO, our standing in the world goes down, Egypt gets overrun by refugees and the Mad Dog of the Mideast, Khaddafy, is out of his cage, and you're going to see oil prices double.
MR. GREGORY: All right. Let me...
SEN. GRAHAM: You will see Iran recalculating.
MR. GREGORY: All right. Let's take a couple of minutes and talk about Afghanistan. This is the other big debate that is really coming to a head this summer. You two are on opposite ends of it in terms of how many troops should come home when this drawdown begins this summer, specifically next month.
But, Senator Graham, it's very interesting because, within the Republican Party, there is a growing debate about Afghanistan as well. Crystallized, Mitt Romney, you said he's the front-runner in the Republican race in the past and he said this...
SEN. GRAHAM: Right.
MR. GREGORY: ...about the America's role in Afghanistan during the debate this week. Listen.
(Videotape, Monday)
FMR. GOV. MITT ROMNEY (R-MA): But I also think we've learned that our troops shouldn't go off and try to fight a war of independence for another nation. Only the Afghanis can win Afghanistan's independence from the Taliban.
(End videotape)
MR. GREGORY: To which you said he may be another Jimmy Carter. You said this about another Republican.
SEN. GRAHAM: Well, I said that the debate last night was more like Jimmy Carter than Ronald Reagan. If you think the pathway to the GOP nomination in 2012 is to get to Barack Obama's left on Libya, Afghanistan and Iraq, you're going to meet a lot of headwinds. This is not a war of Afghan independence from my point of view. This is the center of gravity against the war on terror, radical Islam. It is in our national security interest to make sure the Taliban never come back. If we fail in Afghanistan, they will kill every moderate who tried to help us, and no one in the future will, will step up. It will destabilize Pakistan beyond what exists today. It will be a colossal national security mistake. And here's the good news. The Afghans do not want to be ruled by the Taliban. We've doubled training capacity since December 2009. There'll be 305,000 Afghans under arms.
MR. GREGORY: Well...
SEN. GRAHAM: We're making great progress, 6,000 people a month are joining the Afghan army. In September 2009...
MR. GREGORY: Well...
SEN. GRAHAM: ...it was 800. We're on track to do--to withdraw.
MR. GREGORY: Are you fearful that there is an isolationist streak now running through the Republican Party?
SEN. GRAHAM: Yes. I'm fearful that...
MR. GREGORY: It's not war weariness?
SEN. GRAHAM: ...there are forces on the right...
MR. GREGORY: It's not a practical response to a decade of war?
SEN. GRAHAM: It's not an understanding of what's going on in Afghanistan and the consequences of losing. Are we at war and, if we are, with you? We're at war with radical Islam. That fight is now in Afghanistan. General Petraeus has put the enemy on their heels. We're doubling the capacity to train the Afghan troops. We can bring some troops home this summer. By 2016, if we do this right, five years from now, we can have troops in Afghanistan and Iraq, the same number as South Korea. If we accelerate withdrawal right now because we're war weary, we're going to lose this war.
MR. GREGORY: Let me give you the final word, Senator Durbin. There is a split, based on my reporting, between the military and some Republicans and even some of the thinking within the White House about how to draw down this number. What, in your mind, is an appropriate and sizeable withdrawal from Afghanistan? You were very clear in a letter this week saying, after bin Laden, after the diminishment of al-Qaeda in Afghanistan, it is time to make a sizeable withdrawal.
SEN. DURBIN: Yes. Senator Merkley circulated that letter, and I believe 26 or 27 of us signed that, to the president. I can tell you, David, when I voted 10 years ago against the invasion of Iraq, there were 23 of us. I voted for the invasion of Afghanistan to go after those responsible for 9/11. I didn't vote for the longest war in American history. I didn't vote for 100,000 troops 10 years later in Afghanistan and a notion that somehow we're going to build this into some nation that's a democracy. What we need to do is to have enough security on the ground in Afghanistan that we can bring our troops home. The president made that promise. We should keep that promise. Bring these troops home and let the Afghans deal with the future of their country. The United States cannot literally go from one country to another around the world with all the instability and say that, ultimately, our men and women in uniform will put their lives on the line for the stability of every nation in transition.
MR. GREGORY: All right. I'm going to leave it there. The debate continues. Senators, thank you both very much.
SEN. DURBIN: Thank you.
SEN. GRAHAM: Thank you.
MR. GREGORY: Coming up, Decision 2012. The campaign heating up. Now after two debates for the Republicans, our new NBC News/Wall Street Journal poll out this week shows that Republicans really think the current field is maybe not everything it could be. What are those views and how will Democrats now refocus their agenda after the Weiner scandal? Plus, Moammar Khaddafy still in power, as we've been talking about in Libya, despite persistent airstrikes. What is the endgame for the U.S. and its allies? NBC chief foreign correspondent Richard Engel, just back from the region, is here with us in studio, rare appearance. He'll join our roundtable with the mayor of Los Angeles Antona Villaraigosa--Antonio Villaraigosa and The Wall Street Journal's Paul Gigot, presidential historian Doris Kearns Goodwin, and NBC's Chuck Todd.
MR. GREGORY: Coming up, has Mitt Romney established himself as the front-runner for the GOP presidential nomination? The roundtable is here, ready to weigh in on this week's 2012 news. Plus, the latest developments in the Middle East. We've got Richard Engel here, he's here from the Middle East. Chuck Todd, Doris Kearns Goodwin, Mayor Antonio Villaraigosa and Paul Gigot, they're coming up next right after this.
MR. GREGORY: We're back now, joined by our political roundtable. Chief White House correspondent, political director for NBC News Chuck Todd. NBC's chief foreign correspondent Richard Engel.
Great to have you in person, in studio.
MR. RICHARD ENGEL: Thank you very much. Good to be here.
MR. GREGORY: We can't wait to get your analysis. Thanks, Richard.
(Todd touches Engel's arm)
MR. ENGEL: Hey, you wanted to touch me?
MR. CHUCK TODD: Just want to make sure you're here.
MR. GREGORY: Yeah, he's actually here.
MR. TODD: I've heard of you.
MR. GREGORY: Editorial page editor for The Wall Street Journal, Paul Gigot is here. Presidential historian Doris Kearns Goodwin. And last but not least, my hometown mayor--and I just pronounced your name wrong, which is really embarrassing--the mayor of Los Angeles. Antonio Villaraigosa, for his first appearance here on MEET THE PRESS. Villaraigosa was elected in May of 2005 after serving in the California State Assembly and at LA City Council. Now in his second term. He'll be sworn in tomorrow as president of the U.S. Conference of Mayors in Baltimore. In that role he will represent an organization of more than 1200 mayors and speak out on policy issues impacting cities across the country, and, indeed, some of the big national issues, as well.
Mr. Mayor, welcome to MEET THE PRESS. And, of course, I may pin you down on what is going to happen to our Los Angeles Dodgers in terms of future ownership because there's more than a few very interested parties here at the table. But we're going to get to that in a few minutes.
MAYOR ANTONIO VILLARAIGOSA: Good morning.
MR. GREGORY: Welcome to everybody. Let's talk politics. Let's talk about the race for 2012 and the Republican field. Here is our, our board of who's in right now. And a couple of notes you pay attention to there. Jon Huntsman is going to announce on Tuesday. And, of course, Michele Bachmann, who had a strong debate, has filed papers this past week to get into the race. She'll be doing more of that this coming week. And this is how the field now looks, if you look at our latest NBC News/Wall Street Journal polling. Mitt Romney still on top, Sarah Palin, Herman Cain at 12 percent. And, excuse me, let's just go back to that previous board if we can do it. I glossed over and I shouldn't have, because you've got those who are undeclared or undecided. Palin is the perpetual one that we speculate about; Giuliani; Rick Perry creating a lot of buzz, which we'll talk about in a minute. And even John Bolton, who doesn't think the Republicans are talking enough about foreign policy. You heard that debate just a moment ago with Lindsey Graham.
So, Chuck Todd, there's a lot on the table. Where are we at the end of this week on the Republican side?
MR. TODD: Well, first I'm going to stick with the baseball theme, and that is the focus, I think, is on the player to be named later at this point. Because of the people running right now, Mitt Romney is not just a front-runner. I think he's a very solid front-runner when you look at all of the things that matter to being a front-runner right now, all of the candidates chasing him seem to be stepping on themselves. Tim Pawlenty had a particularly bad week. Michele Bachmann doesn't seem as if she can get the nomination. You can see how she can get very, very strong, and in the end, the Mitt Romney folks love the idea of it being Michele Bachmann. But this idea of the player to be named later, specifically Rick Perry, I think when you look at how this week played out, as good of a week as Mitt Romney had as being a front-runner, you're seeing it in our own poll, you're seeing it in other polls, Rick Perry had a pretty good week, too, because of two things. One, the Tim Pawlenty flubs.
MR. GREGORY: Right.
MR. TODD: And, and, two, the fact that there does seem to be this vacuum here of the anti-establishment wing of the party who's not enthused about who's all out there.
MR. GREGORY: And we'll come--I want to unpack that Tim Pawlenty baggage in just a little bit.
But, Paul Gigot, let me ask you this, Rick Perry, the buzz about Rick Perry, he spoke this weekend at a gathering of, of Republicans down in New Orleans. Here's a portion of what he said.
GOV. RICK PERRY: Let's speak with pride about our morals and our values and, and redouble our effort to elect more conservative Republicans. Let's stop this American downward spiral.
(End videotape)
MR. GREGORY: Here's a, here's a Republican saying, "Let's talk about social issues. Let's talk about moral values." He sees that there's a space here for a social conservative candidate in the race.
MR. PAUL GIGOT: But his trump issue will be jobs.
MR. GREGORY: Right.
MR. GIGOT: I mean, there's a, the Dallas Fed has recently had a, Federal Reserve recently had a study showing that Texas has created 40 percent, nearly 40 percent of all the new jobs since the recession ended. That's going to be his big selling point if he gets in. Yes, he's, he's going to talk about the social issues, but there's plenty of other candidates who are talking about that. I think if he gets in, he'll say, "I'm a governor. I've been in for a decade. I've accomplished a lot. The Texas economy is growing where the U.S. economy is kind of stalling. So what's going to happen is he's, that's the theme he's going to make. The question I have for Perry is, is the country, and particularly those Northern suburbs, ready for another Texan after eight years of, of George...
MR. GREGORY: It's interesting...
MR. GIGOT: ...George W. Bush?
MR. GREGORY: ...Doris, if you look at the debate this week, some of the big headlines, we'll put them on the screen, showed a theme. Focus on Obama and the economy, targeting the president at the debate, target Obama, hits the president on the economy, and on and on it goes. This was, let's not disagree with each other. Let's focus on the president's performance on the economy.
MS. DORIS KEARNS GOODWIN: And in a certain sense, that's what Pawlenty got lambasted for because he was suppose to have taken on Romney on Obamacare. You know, I think some of these things, when we say at the beginning, "Oh, they've undone themselves," we forget how early it is. I remember, this time in '07, McCain was in the wilderness. His campaign was imploding. So we've got a lot way to go. The problem with Pawlenty focusing on Obama and not focusing on Romney at that time was then you have to ask, if he couldn't stand up to Romney, can he stand up to Obama? And you're looking for that forcefulness.
MR. GREGORY: Well, then let's talk about Pawlenty for just a second because in the debate he decides not to take him on on health care in Massachusetts, which he had done in an interview. Then he came on Fox News this week and said, "Hey, I made a mistake." Even said--he called Romney a co-conspirator with President Obama on health care. This is what he said to Fox News.
Mr. TIM PAWLENTY: I should have been much more clear during the debate, Sean. I don't think we can have a nominee that was involved in the development and construction of, of Obamacare and then continues to defend it.
(End videotape)
MR. GREGORY: May...
MS. GOODWIN: You see, I think, though...
MR. GREGORY: Yeah.
MS. GOODWIN: I think the one thing Romney did well in that debate, he has an answer, finally, for the Obamacare thing by saying, "I did it in my state. There are problems in my state. He should have come and asked me what worked, what didn't work." He was prepared for this in a way that he wasn't prepared four years ago. It really helps to have run for the presidency once before. It's a whole different planet. We were talking about planets before.
MR. GREGORY: Yeah.
MS. GOODWIN: And he's learned...
MR. ENGEL: (Unintelligible)...too.
MS. GOODWIN: Exactly.
MR. ENGEL: We were listening to the clip that you played earlier by Mitt Romney and he, there was one thing that jumped out at me that implied a specific lack of preparation.
MS. GOODWIN: Oh, yes.
MR. ENGEL: He--you played a clip earlier and he said, only the Afghanis can decide their political future. That's a concept, I think, everyone would agree with. The problem is, the people are not called Afghanis. They're called Afghans. So if you're try, the Afghani is the currency.
MR. GREGORY: Mm-hmm.
MR. ENGEL: So if you're trying to talk about what the foreign policy should be for a foreign nation...
MS. GOODWIN: He should learn the words.
MR. ENGEL: ...where we have troops...
MR. GIGOT: He should at least...
MR. ENGEL: ...you should know what the people are called.
MR. GREGORY: And I want to come back on this foreign policy points, but I want to hear from the mayor, as well. What is your take? I mean, here you are in a big Democratic state, in, in a city that is dealing all the major issues, from the economy to education reform. How do you see the way the debate is shaping up on the Republican side?
MAYOR VILLARAIGOSA: Well, I think the term was used now twice, it feels like these people are on another planet. I mean, the fact is America's out of work, too many people haven't been able to get back into the workplace. We're not doing enough to train those new workers. We're cutting infrastructure and transportation. In America's cities, we're, we're saying that America needs to focus at home again. And, and this issue of the war in Afghanistan, Libya, Iraq is now an issue that's front and center of debate in our cities.
MR. GREGORY: Well, and you even talked about this in the gathering of mayors. We'll put it on the screen. The New York Times reported on a resolution that you're talking about: "Mayors See End of Wars as a Fix for Struggling Cities. When downturn-weary mayors from around the country gathered ... Friday for the annual meeting of the United States Conference of Mayors ... they introduced a resolution calling for the speedy end of the wars in Iraq and Afghanistan, and calling on Congress to use $126 billion a year the wars cost for urgent domestic needs." You see missed priorities here.
MAYOR VILLARAIGOSA: Absolutely. You know, you have Democrats who don't want to address entitlements and Medicare and Social Security. You have Republicans who say that defense spending is off the table. So what's left? It's infrastructure, it's transportation, it's education, it's public health, it's eviscerating Medicaid and the safety net. And, you know, we actually have to represent these people. We don't live in the bubble of the Beltway. And I think for many of us, although we're about 80 percent Democrat, we're also very practical. And so the issue of pension reform is an issue that almost all of us have embraced. The issue of education reform, seniority and tenure, we're tackling these tough issues. And so we find, and I find, that the debate right now among the Republicans is so out of touch with everyday people who live on Main Street.
MR. GREGORY: Well, let me get an aspect of that, Richard, in terms of how the rest of the world is watching this campaign develop and this internal fight on the Republican side about Afghanistan, about the future of America--America's involvement in the rest of the world, whether it's Libya or Afghanistan. There's real war weariness here.
MR. ENGEL: There, there certainly is. And I think that's understandable. I mean, 10 years the U.S. has been engaged in two major land wars in Afghanistan and Iraq. But I think this debate about Libya, for example, I just came from Libya before I came here, and the fact of the matter is the war in Libya right now is not very serious, that NATO is not doing a terribly good job. The rebels need a lot more help. The bombing campaign in Tripoli barely exists. Every once in a while there's a few bombs on mostly empty compounds, and people go about their lives more or less unaffected. It's not the kind of thing that's going to drive Khaddafy from power. And a, a lot of European nations who are now trying to lead this, this fight, which--and are, and are struggling to do it, are looking at this debate in--within the--in the United States to end the U.S. support for NATO. If the U.S. ended its support for NATO in Libya, NATO really is dead. And all of these European allies, particularly Italy, who supported the United States in Iraq, which they, they had no interest in, would say, "Well, why isn't the United States supporting us now in Libya?"...
MR. GREGORY: Right.
MR. ENGEL: ...which is an important mission for Europe.
MR. GREGORY: Paul, how do you see this debate within the Republican Party? I mean, Lindsey Graham couldn't have been stronger, saying...
MR. GIGOT: It's...
MR. GREGORY: ..."Look, you're going to start taking the president on the left on foreign policy? That's not going to go well in the Republican Party."
MR. GIGOT: There's war weariness, and it's understandable. But I also think, and I regret to say this, there's some real political opportunism here. I mean, you have a Democrat in the White House.
MR. GREGORY: Yeah.
MR. GIGOT: This is not what Republicans would be saying--most of them, nearly all of them--if George W. Bush were still in the White House. So the question, I think, that--fair to ask of a Michele Bachmann, who voted for the Dennis Kucinich resolution to cut off funds within 15 days for Libya, is if you're, you're, you're running to be commander in chief, all right, do you really want as a candidate to invite Congress to micromanage and be able to cut off your use of force if you become president? That's not a very thoughtful position. Yes, it is what Barack Obama did when he was a candidate, but now you can see it's hurting him as he's president. So I think there's some real contradictions here on the--from some of the Republican candidates. And you see the divide between the McCain-Lindsey Graham wing...
MR. GREGORY: Right.
MR. GIGOT: ...and some of the more grassroots, and I would even say somewhat isolationist, elements of the Republican Party.
MR. GREGORY: It's interesting, Doris, though, on this question of leadership around the world, Robert Gates, he's the departing secretary of Defense, he's been giving interviews, and he gave one to Newsweek. And he's been quite outspoken about his views, whether it's about the wars or about how Congress is behaving. We'll put a portion of it up on the screen, the interview that he gave, and it's quite interesting. He said, "I've spent my entire adult life with the United States as a superpower, and one that had no compunction about spending what it took to sustain that position. ... It didn't have to look over its shoulder because our economy was so strong. This is a different time." He says, "To tell you the truth, that's one of the many reasons" for me that I think it's time" to retire, because frankly I can't imagine being part of a nation, part of a government ... that's being forced to dramatically scale back our engagement with the rest of the world. ... Congress is all over the place ... and the Republicans are a perfect example. ... I think there is no consensus on a role on the world." That's a very interesting statement.
MS. GOODWIN: Well, and it comes with such credibility because he's conducted himself so incredibly well during his tenure as secretary. I mean, I think we are confused right now. I mean, I think the problem with Libya right now is we should be debating exactly what Richard said, what should we be doing there substantively? Instead, we're debating president vs. Congress.
MR. ENGEL: Who gets to call the shots.
MS. GOODWIN: And who...
MR. GREGORY: Right, the fight behind the fight. Right.
MS. GOODWIN: And, and it's always going to be a dispute. I mean, the Constitution deliberately made it that way. The president's the commander in chief, but Congress can declare war. So we're going to have these fights forever.
MR. ENGEL: I think you're going to see more of them going forward because small wars, after this last decade...
MS. GOODWIN: They become--that's right.
MR. ENGEL: ...are going to become the future.
MS. GOODWIN: The really interesting thing is James Baker wrote...
MR. ENGEL: Drone wars, little wars like Libya.
MR. GREGORY: Right.
MS. GOODWIN: James Baker wrote an editorial in The Washington Post because he was part of a commission that I was an historical adviser for, a war powers commission, arguing that the War Powers Act is unconstitutional, it needs to be repealed. What we need to replace it with is a war consultation act where, before you go to one of these things, you go to the Congress. You should go to the Congress. It helps you to communicate to the country. But if it's something like bin Laden, you don't because it's secrecy. It doesn't work this way to have to do it afterwards.
MR. GREGORY: Well, part of it's about, again, what are we doing? What's, what's, what--if the rebels take power, what does that look like? And how long does Khaddafy stay?
MR. TODD: Well, look, a couple things. One is, is, the one thing, that every president believes the War Powers Act is unconstitutional.
MS. GOODWIN: Right.
MR. TODD: And if you notice in the very ways they word these legal memos--Bob Bauer, the current White House counsel, very specific about saying certain things are consistent with the idea of the War Powers Act, you know, because they don't--most presidents don't want to acknowledge that--the...
MR. GREGORY: Mm-hmm.
MR. TODD: But let's also talk about why this is also a problem. The president has done poor personal politics on selling Libya, period. OK? When you've got Jim Webb out there, who would be a national security Democrat, spend a little time with him, explain to him the mission. Dick Lugar, Republican, spend a little time with him, explain the mission. He's not done the personal politics of this. Joe Biden usually--it's usually outsourced to Joe Biden, the way this White House works. He's been in the middle of the debt negotiations and this whole debt commission. So I think, yes, you have Congress all over the place. Yes, there's all these issues with, with the Republican Party inside Congress right now. And yes, I do think there's some opportunism. But the president hasn't made a good faith effort at just good, old-fashioned back-slapping politics.
MR. GIGOT: And I would add, not only, not only within Congress, but to the American public. I mean, he doesn't not want to talk about Afghanistan or Libya because I think they feel that the--in the White House, he needs to focus only on domestic issues. There's only downside on foreign policy. But if you're going to send American forces into battle, you have to make the sale.
MR. GREGORY: All right, I'm going to take a break here. Couple things I want to do when we come back: talk about the final act in this Weiner scandal, more about the economy and how that is going to shape the debate for the presidency, and more about the Republican approach to it all. More from our roundtable right after this.
MR. GREGORY: We're back now with more from our roundtable.
Doris, a big story still this week, Anthony Weiner. He did finally resign. One of the questions that comes up is why it is some politicians can have sex scandals that they can survive and yet he couldn't. What happened here?
MS. GOODWIN: Well, I think a couple of things happened. One was the timing of it. For the Democrats, it was disastrous. They were just making a momentum on the fact that they had won the special election, but yet another congressman who had to resign because of an inappropriate thing on the Internet. And they felt like he distracted them from that news. So they were putting pressure on him. Secondly, he lied to his colleagues, to the country, to the press. And thirdly, I think the saturation of the media today is such that he must've felt like a mountain was falling on him. The only way to relieve that pressure was to get out.
MR. GREGORY: And, Chuck, he just didn't have the support. I mean, he, he called Nancy Pelosi and said look at my polling. It's still holding up in my district. She said, take that as a sign that you can bow out gracefully here.
MR. TODD: Well, no. What it--it's a message to some of these members of Congress saying it's good to have friends.
MR. GREGORY: Yeah.
MR. TODD: And he found out he really didn't have any friends, and sort of the way he might butt into a press conference or do some things, clearly had irritated Nancy Pelosi over the years. Nobody felt the need to go out there. What I just didn't understand is why he thought having yet another press conference, going in front of the cameras one more time was a good idea. It seemed it only fed the worst parts of this scandal. You're right about the media saturation. And by the way, I don't think in this environment, anybody--I don't think David Vitter would've survived...
MS. GOODWIN: In this environment.
MR. TODD: ...in this environment.
MS. GOODWIN: What's sad about it, though, is we waste so much time in our political arena.
MR. ENGEL: I was shocked by how much time is wasted on this kind of stuff...
MS. GOODWIN: We took three weeks on Weiner.
MR. ENGEL: ...when I got back.
MS. GOODWIN: They spent them three weeks...
MR. ENGEL: Now I know why Osama bin Laden has his messages hand delivered.
MR. GREGORY: Right.
MS. GOODWIN: But I mean, three weeks on Weiner, three weeks on the birth certificate. We've got serious problems, as you were saying.
MAYOR VILLARAIGOSA: Right.
MS. GOODWIN: And we're not focusing on that.
MAYOR VILLARAIGOSA: Yeah.
MR. GREGORY: Mayor, let me turn to that. On the, on the economy specifically because, again, front line mayor, dealing with these problems. Here's the cover of Newsweek, and it's former President Clinton who, lo and behold, has a few things to say about the economy, "14 Ways to Save America's Jobs." And "It's Still the Economy, Stupid,' it says inside. Can you make a case for Democratic governance given the economic woes of this country that have persisted under President Obama?
MAYOR VILLARAIGOSA: Without question. But let me say something about the three weeks of scandal that you referred to. For most of us watching that circus and, and the amount of time and air it had consumed, it--again, feels like you're on another planet. People are losing their homes due to foreclosure. People are out of a job in, in double digit numbers. According to our metro economies report, we're not going to get back to where we need to be for another 10 years in some cities. And so, yes, I think you can make a case that, that with Republicans primarily threatening to default and what that would mean to the economy, what it would mean to, to an increase in unemployment, I think you could say that most of--when, when Bush inherited a surplus and, over an eight-year period of time, took us to the highest deficits and debt in our history, you've got to say call it like you see it. And the Republicans certainly have put us here.
MR. GREGORY: But, Paul Gigot, you can't keep blaming Bush. I mean, there, there are just some stubborn figures.
MR. GIGOT: The economy--yeah, I don't think it worked in 2010 very well. And I don't think it's going to work in 2012. President Obama owns this economy. And, yes, he can go and try to, "I inherited a mess," and I think even some of the candidates would say yes--Republicans would concede, "Yes, you did." But the question is, did you make it better or did you make it worse? And if not--if unemployment a year from now is 9.1 percent or even 8.5 percent and the deficit is still a trillion dollars, and you don't see--and you see growth at 2 percent, the president's going to have a very hard time winning re-election.
MR. TODD: You know, David, the--there, there's something bigger that I've found from our poll that I think we're all missing here, which is you can see the collective frustration on the American public again at Washington. Congress hit its low that it hasn't hit since March 2010, the healthcare month. You have people saying they want government to "do more," but at the same time, they have 60 percent of them say, "Will you please do something about this debt and deficit mess? We get it on the economy." They concede that it's--that Obama had inherited this mess. They're basically--the message they're sending us, "Just do something." And the worst thing for both parties will be some sort of gridlock, and then look at other countries and how they've handled that. Japan has gone through--they go through a prime minister every six months because of a public frustration with the government's inability to just solve problems.
MR. GREGORY: And you know, Richard, from your vantage point, covering much of the world, the businessmen I talk to in this country who, of course, are traveling around the world say you go to a country like China, they're not talking about America as much as they may be talking about certain European countries. That, that notion of American decline or a diminishing role in the world is something that is--stems from an inability of our political system to meet the challenges we face.
MR. ENGEL: The--they talk about the United States in terms of its military primarily. Not necessarily its economy. People don't--aren't rushing here to do business. It's become difficult. I--primarily coming from the Arab world, is thought of as an unfriendly place, it's hard to get visas, you get hassled if you're here. And they think of us in terms of our U.S. military, which is why--as an organization that's still works and is still an incredibly unique capability. Which is why I'm finding that comment that you played earlier by Secretary Gates so incredibly chilling, that he's basically saying, "Our economy is so bad right now that we are not in a position--and our political leadership is so fractured--that we're not in a position to lead the world any longer and I don't want to be part of that." And that's frightening.
MR. GIGOT: Yeah.
MR. GREGORY: Doris, we, we showed the pictures from Greece. We can show them again demonstrations in the streets as a result of draconian cuts being made by the government there. The same question I asked to Senator Graham, are we being too aggressive about tackling deficits at a point when the economy still needs so much help?
MS. GOODWIN: Absolutely. I mean, it seems to me the rational answer right now, we've got to get the economy to grow, and then get some problems which we agree on both sides that we will have targets for cutting the deficit. I mean, most economists would argue that. But somehow the deficit has become--it's the same way it became with Perot, it's become that lady in the closet. We have to deal with it, we do have to deal with it.
MR. GREGORY: Right.
MS. GOODWIN: But you've got to deal with the economy first, get it going, and then you make promises about getting that down.
MR. GREGORY: But even--I've spoken to House Republican leaders who say, you know, this is--even our zeal to cut the debt...
MS. GOODWIN: Mm-hmm.
MR. GREGORY: ...is hurting our ability to have a jobs agenda...
MS. GOODWIN: Yes.
MR. GREGORY: ...to have some positive ideas to say how do we get people back to work?
MR. GIGOT: Yeah. There's a little bit of an accountant's mentality that comes across, I think, and that's a problem. But I--I think, philosophically, from an economics policy point of view, you cut spending, you're going to help the economy, in my view. Because every dollar the government spends is a dollar that's taken out of the private sector, which is the only place, the only place you're going to drive growth. But the problem for the Republicans is a little too much talk about budget balancing and not enough talk about growth and jobs and getting growth back up to 3 and 4 percent.
MR. GREGORY: Who's hiring? Can I...
MAYOR VILLARAIGOSA: Or investment in infrastructure.
MR. GREGORY: Yeah.
MAYOR VILLARAIGOSA: Or investment in education or investment in workforce development. I mean, there is no conversation from that side of the aisle. And don't get me wrong, I'm not here to defend Democrats. I think your point was a good one. People are tired of the gridlock, the partisanship, the polarization, the shrill debate. I mean, we're, we're excited that the president and the speaker are on some, you know, golf tournament day. You know, the fact of the matter is, they should be talking on a regular basis, and they're not talking. All they're doing is screaming at one another.
MR. GREGORY: Mm-hmm.
MR. ENGEL: You talk about money the U.S. spent fighting this global war on terrorism. I think, which is a terrible misnomer, it's like a war on fear or something like that. And I think in many ways it has been a war of fear. But the United States spent at least a trillion dollars on this for the last decade. And we've become very good at killing bearded men in Waziristan. But there are trade-offs. We've--China owns a tremendous amount of our currency, and we've focused on other--ignored other national priorities, so...
MR. GREGORY: All right. We're going, we're going to get a break in here. We'll come back with our final segment, Trends & Takeaways, a look at what was said here today and what to look for in the coming week. Plus, what are the hot political stories trending this morning? That's coming up, right after this.
MR. GREGORY: We're back. Final minutes with our roundtable here. And to review, Senator Lindsey Graham talking about what he sees as an isolationist streak within the Republican Party over foreign policy, particularly responding to the debate this week in New Hampshire, said earlier on this program this following:
SEN. GRAHAM: If you think the pathway to the GOP nomination in 2012 is to get to Barack Obama's left on Libya, Afghanistan and Iraq, you're going to meet a lot of headwinds. This is not a war of Afghan independence, from my point of view. This is the center of gravity against the war on terror.
(End videotape)
MR. GREGORY: Paul Gigot, quickly, where are the cross-currents in the Republican Party? Does Graham win out or does the tea party, the focus on the debt and spending ultimately win the day, which means more war weariness?
MR. GIGOT: I think probably Graham wins out in the end, just because I think that the, the tradition, the modern tradition of the Republican Party tends to be a strong military party, and that tradition will be hard to, to beat. To really get an isolationist trend in the GOP, you've got to go back to the '50s, before Eisenhower, or back to the, to the, to the, well, Woodrow Wilson.
MR. GREGORY: Mm-hmm.
MR. GIGOT: So I think that, I think in the end Graham will win out, particularly if the president starts to make the sale on, on Libya and Afghanistan.
MR. GREGORY: Foreign policy is still on the top three on our political trend tracker. The top stories on the big political Web sites that we'll put up on our screen, our political trend tracker showing you, it's Rick Perry, actually, at number one, getting buzz from this weekend's gathering of Republicans, the Obama-Boehner golf summit. These are stories that are still gaining attention. Libya and the war powers debate losing a little bit of steam in terms of people focusing on it, but it's still in that top three.
Now, Chuck Todd, to you. As we look ahead on the political schedule, not just for this week, but really a summer of intense campaigning, what are you watching out for?
MR. TODD: There's two dates that I'm watching in particular, and number one is July 15th. That is the FEC fundraising deadline. We're going to find out, rubber hits the road on money. How much did Mitt Romney raise? You know, you had somebody hinting from his dream that they think he can get to $40 million to $50 million, a very Bush-like number when Bush was running in '99. What's Tim Pawlenty's number? Does Michele Bachmann outraise him? That's going to send a message to Rick Perry. If Romney has this insurmountable financial lead...
MR. GREGORY: Right.
MR. TODD: ...Perry may bow out on this.
MR. GREGORY: All right. And go through some of the other days, quickly.
MR. TODD: And then I would say here August 6th is Rick Perry's National Day of Prayer that he's hosting.
MR. GREGORY: Mm-hmm.
MR. TODD: Reliant Stadium, home of the Houston, Texas, in Houston. That's going to tell us a lot. He's not going to announce before that date. I've been told by advisers he doesn't want to politicize that thing, but it's going to tell you, this is a guy that can speak to the tea party wing, the social conservative wing...
MR. GREGORY: Mm-hmm.
MR. TODD: ...talk jobs, as Paul mentioned. He can put together two legs of that stool.
And then finally, I think the, the Iowa Straw Poll. It's going to tell us whether the Tim Pawlenty campaign is for real or not.
MR. GREGORY: All right. Interesting.
Talking about campaigns, we noticed something on Twitter today from President Obama, and specifically from his campaign for next year. On their campaign logo they've made it clear, as we've seen today, from Barack Obama it says, "Welcome to a new @BarackObama. ... staff will manage this account; tweets from president will actually be signed `-BO.'" So you know they're actually coming from him. So...
MS. GOODWIN: That's the modern world.
MR. GREGORY: The modern world. Everybody these days...
MR. TODD: But he hasn't tweeted yet though, right? Even he himself had to have been, "What's Twitter?"
MS. GOODWIN: I would think that after...
MR. GREGORY: All right.
MR. TODD: He didn't quite know how to do it.
MR. GREGORY: All right. We're going to leave it there.
Thank you all very much for the discussion today. A happy Father's Day to mine and to yours. We'll see you next week. If it's Sunday, it's MEET THE PRESS.


text:
Huddinge Municipality assigned the names of two roads passing through BoKlok Visättra Ängar and BoKlok Solhöjden. The main road which pass from all four Område is called "Småbruket backe". On the right side it connect with Karnängsvägen and on the left side it join with Mangårdsvägen. The name "Småbrukets" came from old town map of 1947 which indicate… Read more →
Tag: Skanska
Visättra Ängar – December 2014
In December all the BoKlok Visättra Änger apartments were build and assembled and for BoKlok Solhöjden the base of C block was also laid down. But even there is lot of work t be done e.g. roads, parking space, balcony and patios are still to be started. BoKlok also send the invitation for "Upplåtelseavtal" lease agreement meeting for 3rd of… Read more →
Visättra Ängar – November 2014
During November a lot of development were carried out in BoKlok Visättra Ängar and BoKlok Solhöjden. BoKlok also had a visit (tomtvisning) on 22nd November at 11:00. On the Visit day weather was little cold but Korvgrill served by BoKlok was a nice touch, it helps the guest to warm up for the visit. Read more →
Visättra Ängar – October 2014
Today i paid a visit to BoKlok Visättra Ängar to look at the construction progress. And i was very pleased to see that lot of ground work was already complete. All the underground cabling for apartment were laid and all the basses for apartment were complete too, plus the Storage rooms for Block/Hus D,E, F and G were also completed.… Read more →
BoKlok Solhöjden
BoKlok Solhöjden, Huddinge is the 2nd development carried out by the BoKlok in the vicinity of BoKlok Visättra Ängar. In the original bidding process by Huddinge municipality marked both of these development are branded as one area "Område 4". BoKlok Solhöjden is a BoKlok Flex style flats and consists of 48 apartments on four floors. The Draw of apartments was… Read more →
Boklok Visättra Ängar – Construction site during August 2014
My visit to Visättra Ängar in early august, Skanska crew were there and certainly some work were been done, Temporary housing setup for construction workers were already build and heavy machinery were also present. It was easily recognizable that work is going on for preparing and leveling the land, instead of constriction at this stage.


text:
It seems we can't find what you're looking for. Perhaps searching can help.
It seems we can't find what you're looking for. Perhaps searching can help.
There are 107 miles of mountain bike trails in the area. Mountain bikers of all abilities will have a good time exploring 3 beginner trails, 6 intermediate trails, 1 easiest trails.
Know of a new mountain bike trail not listed here? Please tell us about it here.
Singletracks members have ranked the best mountain bike trails in Omaha, Not set using our unique trail ranking algorithm. The best bike trails are rated by weighing the following ranking factors: average trail rating, number of members who have ridden the MTB trail, and the number of members who want to ride the MTB trail. Disagree with our list of the top mountain bike trails in Omaha, Not set? Find and review your favorite Omaha, Not set bike trails. Constantly updated.


text:
0 Members and 1 Guest are viewing this topic.
Anne,Last time I was in Paeroa (probably October) there was a most beautiful bright blue one of these bowls for sale at one of the junk shops.Unfortunately I had spent all my pocket money!CheersMeegs
Yes it's on eBay Tony. £30+postage
Hi Meegs and AnneI was in Paeroa with my wife today Friday 20/01/2012 The blue bowl is still there a real beauty but at $120 I left it there (it is about 20cm in diameter)Tony H
Page created in 0.85 seconds with 27 queries.


text:
Ink cartridges for Brother printers
Using original Brother consumables guarantees optimum performance.
Easy to replace.
Low cost per copy.
Optimised image reproduction.
Excellant in draft mode.
Eliminate the frustration of throwing away unused inks - Seperate ink cartridges for each colour helps reduce wastage as you only replace the empty colour cartridge.